# Inaya Labs — Developer Docs (llms-full.txt)

> Plain-text mirror of every guide and tutorial, generated for LLM/agent
> consumption by tools/build-docs.mjs. Do not edit — edit the Markdown in
> docs-src/ and rebuild. Canonical HTML: docs/index.html

## Guides (User Journeys)

### G-00 · Your first API call in 5 minutes
URL: /docs/guides/first-api-call.html

> Pick an environment, prove a service is alive, present your key pair, and make one safe authenticated call.

> **Journey.** As a developer (human or agent-operated), I want to reach a running
> Inaya Labs service and make one authenticated request, so that I can confirm my
> environment before building anything real.

Every integration starts the same way: pick an environment, prove a service is alive,
present your credentials, and make one small authenticated call. This guide walks that
path end-to-end. Nothing here writes data.

## Prerequisites

- One of the environments below is reachable from your machine (local dev stack, or the
  shared dev subdomains).
- An API **key pair** (`x-api-key` / `x-client-app-key`) issued for your tenant. For the
  shared dev environment, request the beta pair from your Inaya Labs contact; for a
  local stack, the pair is seeded by the platform migrations.
- `curl`, or any HTTP client. Snippets also show Python and Node.

## Concepts (only what this journey needs)

- **Services are per-purpose microservices**, each on its own port (local) or subdomain
  (dev/prod). There is no single API gateway URL to memorize — you call the service you
  need.
- **Health endpoints are unauthenticated.** Every service exposes one so orchestrators
  (and this guide) can probe liveness without credentials.
- **Everything else is authenticated** with a tenant key pair, and write endpoints also
  require a JWT obtained through the Auth Service (see the Authentication guide).

## Step 1 — Pick your base URL

Snippets use `${BASE_URL}`. Map it to your environment:

| Environment | `BASE_URL` | Notes |
|---|---|---|
| Local dev stack | `http://127.0.0.1:<service-port>` | Per-service ports, e.g. Auth `:40004`, Memcache `:40011`. Always `127.0.0.1`, not `localhost` (IPv6 refusal hangs some clients). |
| Shared dev | `https://<service>-sandbox.inayalabs.com` | e.g. `https://auth-service-sandbox.inayalabs.com` |

## Step 2 — Prove a service is alive

Call an unauthenticated health endpoint. Auth Service:

```bash verify=auto
curl -s "${BASE_URL}/v3/health"
```

```python verify=auto
import os, urllib.request

base = os.environ["BASE_URL"]
print(urllib.request.urlopen(f"{base}/v3/health").read().decode())
```

```javascript verify=auto
const base = process.env.BASE_URL;
const res = await fetch(`${base}/v3/health`);
console.log(await res.text());
```

**Expected response** (the platform `HealthInfo` — minimal by design):

```json
{ "status": "Up" }
```

> [!PITFALL]
> A `000`/connection-refused almost always means one of: the service isn't running, you
> used `localhost` instead of `127.0.0.1` on the local stack, or you're hitting a dev
> subdomain without network access — not an auth problem.

## Step 3 — Present your key pair

> [!NOTE]
> **Sandbox exception.** Services running with `INAYA_LABS_ENVIRONMENT=sandbox` (the
> Playground stack) skip authentication entirely — any or no JWT and any or no tenant
> key pair are accepted, and requests run under a synthetic `sandbox-user` identity.
> The key pair below is only required outside the Sandbox environment.

All non-health endpoints require the tenant key pair as headers:

| Header | Value |
|---|---|
| `x-api-key` | Your tenant API key |
| `x-client-app-key` | Your client application key |

Keep them in your environment, never in source control:

```bash
export INAYA_API_KEY="…"
export INAYA_APP_KEY="…"
```

## Step 4 — Make your first authenticated call

List the federated sign-in methods the tenant supports — a safe, read-only first call:

```bash verify=auto
curl -s -X POST "${BASE_URL}/v3/auth/login_methods" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d '{"locale": "en-US"}'
```

```python verify=auto
import os, json, urllib.request

req = urllib.request.Request(
    f"{os.environ['BASE_URL']}/v3/auth/login_methods",
    data=json.dumps({"locale": "en-US"}).encode(),
    headers={
        "Content-Type": "application/json",
        "x-api-key": os.environ["INAYA_API_KEY"],
        "x-client-app-key": os.environ["INAYA_APP_KEY"],
    },
)
print(json.dumps(json.load(urllib.request.urlopen(req)), indent=2))
```

```javascript verify=auto
const res = await fetch(`${process.env.BASE_URL}/v3/auth/login_methods`, {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "x-api-key": process.env.INAYA_API_KEY,
    "x-client-app-key": process.env.INAYA_APP_KEY,
  },
  body: JSON.stringify({ locale: "en-US" }),
});
console.log(await res.json());
```

**Expected response** — every method the tenant supports, plus the subset
enabled for the request's region (contents vary by tenant and locale):

```json
{ "all_methods": ["FederatedApple", "FederatedGoogle", "Passkey"], "regional_methods": ["FederatedApple"] }
```

> [!PITFALL]
> `401` here means the key pair was rejected: wrong tenant, wrong header names, or a
> stale key. `415` means you forgot `Content-Type: application/json` — every POST body
> in the platform is JSON unless documented otherwise.

## Full working example

A complete script that checks health, then lists login methods — save as `first_call.sh`
and run with `BASE_URL`, `INAYA_API_KEY`, `INAYA_APP_KEY` set:

```bash verify=auto
#!/usr/bin/env bash
set -euo pipefail

echo "1. Health:"
curl -sf "${BASE_URL}/v3/health" && echo

echo "2. Login methods:"
curl -sf -X POST "${BASE_URL}/v3/auth/login_methods" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d '{"locale": "en-US"}'
echo
```

## Try it live

Open the [Playground](../../playgrounds/playground.html) → **Users & Profile Matching** → *Users*
console — it signs in against the live Auth Service and exercises the same key-pair
headers.

## Copy agent prompt

```prompt
You are integrating with the Inaya Labs platform. Environment: ${BASE_URL}
(local or dev per the mapping table in guide G-00). Credentials are in
INAYA_API_KEY / INAYA_APP_KEY env vars (x-api-key / x-client-app-key headers).
First verify ${BASE_URL}/v3/health returns 200, then call POST
/v3/auth/login_methods with body {"locale":"en-US"} and the key-pair headers,
and report the enabled login methods. Do not retry 4xx responses; only retry
connection failures, up to 3 times.
```

## API reference

- [Auth Service — Swagger UI](https://auth-service-sandbox.inayalabs.com/v3/auth/swagger-ui/) · [Scalar](https://auth-service-sandbox.inayalabs.com/v3/auth/scalar)

## Next journeys

- **Authentication & sessions** (G-01, Phase 1) — sign in, handle tokens, sign out.
- **Errors, retries & idempotency** (G-03, Phase 1) — what to do when calls fail.

### G-01 · Authentication & sessions
URL: /docs/guides/authentication-sessions.html

> Discover the sign-in methods a tenant supports, complete a sign-in, handle the token pair, refresh it, and sign out.

> **Journey.** As a developer, I want my users to sign in, hold a session, and sign
> out cleanly, so that my app can act on their behalf without embedding credentials.

The Auth Service is the platform's front door. This guide covers the full session
lifecycle: discover what sign-in methods a tenant supports, complete a sign-in, use
the resulting tokens, rotate them before they expire, and retire them at sign-out.

## Prerequisites

- The Auth Service is reachable (local stack `:40004`, or the shared dev subdomain).
- Your tenant **key pair** (`x-api-key` / `x-client-app-key`) — the same pair from
  guide G-00. All Auth Service endpoints except health require it.
- **Memcache Service** reachable (`:40011` local) — sign-in and refresh store the
  refresh token there; without it both calls fail with `500 Failed to store
  refresh token in MemCache`.
- `curl`, or any HTTP client. Snippets also show Python and Node.

## Concepts (only what this journey needs)

- **Two credential tiers, always together.** The *key pair* proves which tenant
  app is calling (vendor auth); the *JWT* proves which user the call is for (user
  identity). Most endpoints in other services require both; the Auth Service's
  sign-in endpoints require the key pair, and token-gated endpoints (logout) add
  the JWT.
- **Sign-in is method-agnostic.** Passkeys, and federated providers (Apple,
  Google, …) all produce the same result: a token pair.
- **Tokens are short-lived and rotatable.** The access JWT expires in 24 hours;
  the refresh token rotates on every use — a successful refresh returns a *new*
  pair and invalidates the old refresh token.
- **Sign-out retires the JWT** so it cannot be replayed (JWT retirement is active
  when the service runs with `INAYA_CHECK_EXPIRED_JWTS=true`).

## Step 1 — Discover the tenant's sign-in methods

Before showing any UI, ask the Auth Service which methods this tenant supports:

```bash verify=auto
curl -s -X POST "${BASE_URL}/v3/auth/login_methods" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d '{"locale": "en-US"}'
```

```python verify=auto
import os, json, urllib.request

req = urllib.request.Request(
    f"{os.environ['BASE_URL']}/v3/auth/login_methods",
    data=json.dumps({"locale": "en-US"}).encode(),
    headers={
        "Content-Type": "application/json",
        "x-api-key": os.environ["INAYA_API_KEY"],
        "x-client-app-key": os.environ["INAYA_APP_KEY"],
    },
)
print(json.dumps(json.load(urllib.request.urlopen(req)), indent=2))
```

```javascript verify=auto
const res = await fetch(`${process.env.BASE_URL}/v3/auth/login_methods`, {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "x-api-key": process.env.INAYA_API_KEY,
    "x-client-app-key": process.env.INAYA_APP_KEY,
  },
  body: JSON.stringify({ locale: "en-US" }),
});
console.log(await res.json());
```

**Expected response** — everything the tenant supports, plus the subset enabled
for the request's region:

```json
{ "all_methods": ["FederatedApple", "FederatedGoogle", "Passkey"], "regional_methods": ["FederatedApple"] }
```

Render your sign-in UI from `regional_methods`; fall back to `all_methods` when
it is empty.

## Step 2 — Sign in

All flows follow the same shape: **begin** (server creates a challenge) →
**complete with the identity provider** → **finish** (server verifies and issues
the token pair). The middle step depends on the method:

- **Passkey** — `POST /v3/auth/pass_key/login/begin` returns WebAuthn options;
  the browser/platform authenticator signs them; `POST /v3/auth/pass_key/login/finish`
  verifies the signature and returns the token pair. (Registration uses the same
  begin/finish pair under `/v3/auth/pass_key/register/*` — the Playground's Users
  console exercises it live.)
- **Federated** — `POST /v3/auth/federated/login` exchanges the provider's
  identity token (Apple/Google SDK on the client) for the platform token pair.
- **QA environments** — `POST /v3/auth/qa/login` creates a user on the fly and
  returns a real token pair with no external identity provider. It exists for
  integration tests and local development; it is not available in production.

Kick off a passkey sign-in (this first half is automatable; the authenticator
response is not):

```bash verify=auto
curl -s -X POST "${BASE_URL}/v3/auth/pass_key/login/begin" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"device_id\": \"my-device-001\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\"}"
```

**Expected response** — a `challenge_id` plus WebAuthn assertion options (challenge,
allowed credentials) to hand to the client's WebAuthn API (`navigator.credentials.get`):

```json
{ "challenge_id": "…", "options": { "challenge": "…", "rpId": "…", "allowCredentials": [{ "id": "…", "type": "public-key" }] } }
```

> [!NOTE]
> **Manual step.** Pass `options` to the authenticator, then submit the signed
> assertion to `POST /v3/auth/pass_key/login/finish` (body: `challenge_id`, the
> PublicKeyCredential JSON as `credential`, plus `device_id`, `locale`, and
> `tenant_id`). The response is the token pair below.
> Federated sign-in is likewise a manual walkthrough: obtain the provider token
> client-side, then `POST /v3/auth/federated/login` with it. Neither step can be
> executed by a script — that is the point of the credential.

In local and QA environments you can shortcut to a real token pair for testing:

```bash verify=auto
curl -s -X POST "${BASE_URL}/v3/auth/qa/login" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"device_id\": \"my-device-001\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"dev-user\"}"
```

```python verify=auto
import os, json, urllib.request

req = urllib.request.Request(
    f"{os.environ['BASE_URL']}/v3/auth/qa/login",
    data=json.dumps({
        "device_id": "my-device-001",
        "locale": "en-US",
        "tenant_id": os.environ["TENANT_ID"],
        "username": "dev-user",
    }).encode(),
    headers={
        "Content-Type": "application/json",
        "x-api-key": os.environ["INAYA_API_KEY"],
        "x-client-app-key": os.environ["INAYA_APP_KEY"],
    },
)
pair = json.load(urllib.request.urlopen(req))
print(json.dumps(pair, indent=2))
```

```javascript verify=auto
const res = await fetch(`${process.env.BASE_URL}/v3/auth/qa/login`, {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "x-api-key": process.env.INAYA_API_KEY,
    "x-client-app-key": process.env.INAYA_APP_KEY,
  },
  body: JSON.stringify({
    device_id: "my-device-001",
    locale: "en-US",
    tenant_id: process.env.TENANT_ID,
    username: "dev-user",
  }),
});
console.log(await res.json());
```

**Expected response** — the token pair every successful sign-in returns:

```json
{
  "token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.…",
  "refresh_token": "9f1c…",
  "user": { "id": "…", "username": "dev-user" }
}
```

## Step 3 — Hold and present the session

- Keep `token` and `refresh_token` in your app's secure storage (keychain,
  secure enclave) — never in logs, URLs, or source control.
- Send the access token as `Authorization: Bearer <token>` on every user-data
  endpoint, **alongside** the key-pair headers. One without the other is rejected.
- Do not parse the JWT client-side to decide expiry; treat HTTP `401` responses
  as the expiry signal and refresh (Step 4).

## Step 4 — Refresh before expiry

Exchange the refresh token for a fresh pair. Refresh tokens rotate: the old
refresh token stops working the moment this call succeeds.

```bash verify=auto
curl -s -X POST "${BASE_URL}/v3/auth/refresh" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"device_id\": \"my-device-001\", \"locale\": \"en-US\", \"refresh_token\": \"${REFRESH_TOKEN}\", \"tenant_id\": \"${TENANT_ID}\"}"
```

```python verify=auto
import os, json, urllib.request

req = urllib.request.Request(
    f"{os.environ['BASE_URL']}/v3/auth/refresh",
    data=json.dumps({
        "device_id": "my-device-001",
        "locale": "en-US",
        "refresh_token": os.environ["REFRESH_TOKEN"],
        "tenant_id": os.environ["TENANT_ID"],
    }).encode(),
    headers={
        "Content-Type": "application/json",
        "x-api-key": os.environ["INAYA_API_KEY"],
        "x-client-app-key": os.environ["INAYA_APP_KEY"],
    },
)
print(json.dumps(json.load(urllib.request.urlopen(req)), indent=2))
```

```javascript verify=auto
const res = await fetch(`${process.env.BASE_URL}/v3/auth/refresh`, {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "x-api-key": process.env.INAYA_API_KEY,
    "x-client-app-key": process.env.INAYA_APP_KEY,
  },
  body: JSON.stringify({
    device_id: "my-device-001",
    locale: "en-US",
    refresh_token: process.env.REFRESH_TOKEN,
    tenant_id: process.env.TENANT_ID,
  }),
});
console.log(await res.json());
```

**Expected response** — a new pair; replace both stored values:

```json
{ "token": "eyJ…", "refresh_token": "c4aa…" }
```

> [!PITFALL]
> A `403` here almost always means the refresh token was already rotated — another
> concurrent refresh won. Serialize refreshes per device: one in flight at a time,
> and always store the most recent pair the server returned.

## Step 5 — Sign out

Retire the session server-side (requires the access token — note `PUT`):

```bash verify=auto
curl -s -X PUT "${BASE_URL}/v3/auth/logout" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}"
```

```python verify=auto
import os, json, urllib.request

req = urllib.request.Request(
    f"{os.environ['BASE_URL']}/v3/auth/logout",
    method="PUT",
    headers={
        "Authorization": f"Bearer {os.environ['TOKEN']}",
        "x-api-key": os.environ["INAYA_API_KEY"],
        "x-client-app-key": os.environ["INAYA_APP_KEY"],
    },
)
print(json.load(urllib.request.urlopen(req)))
```

```javascript verify=auto
const res = await fetch(`${process.env.BASE_URL}/v3/auth/logout`, {
  method: "PUT",
  headers: {
    Authorization: `Bearer ${process.env.TOKEN}`,
    "x-api-key": process.env.INAYA_API_KEY,
    "x-client-app-key": process.env.INAYA_APP_KEY,
  },
});
console.log(await res.json());
```

**Expected response**:

```json
"Logged out successfully"
```

> [!NOTE]
> When `INAYA_CHECK_EXPIRED_JWTS=true` the logout also records the token's `jti`
> in the retirement store, so the same JWT is rejected even before its natural
> expiry. Always discard both tokens locally regardless.

## Full working example

A complete sign-in → refresh → logout cycle for local/QA environments
(`first_call.sh` pattern from G-00, extended):

```bash verify=auto
#!/usr/bin/env bash
set -euo pipefail

# 1. Discover methods
curl -sf -X POST "${BASE_URL}/v3/auth/login_methods" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d '{"locale": "en-US"}' > /dev/null && echo "methods: ok"

# 2. Sign in (QA path — production uses passkey/federated begin+finish)
PAIRS=$(curl -sf -X POST "${BASE_URL}/v3/auth/qa/login" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"device_id\": \"my-device-001\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"dev-user\"}")
TOKEN=$(echo "$PAIRS" | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])')
REFRESH=$(echo "$PAIRS" | python3 -c 'import json,sys; print(json.load(sys.stdin)["refresh_token"])')
echo "signed in: ok"

# 3. Refresh
curl -sf -X POST "${BASE_URL}/v3/auth/refresh" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"device_id\": \"my-device-001\", \"locale\": \"en-US\", \"refresh_token\": \"${REFRESH}\", \"tenant_id\": \"${TENANT_ID}\"}" > /dev/null \
  && echo "refresh: ok"

# 4. Sign out
curl -sf -X PUT "${BASE_URL}/v3/auth/logout" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" > /dev/null && echo "logout: ok"
```

## Try it live

Open the [Playground](../../playgrounds/playground.html) → **Users & Profile Matching** →
*Users* console — its passkey-registration flow exercises the begin/finish
contract against the live Auth Service.

## Copy agent prompt

```prompt
You are integrating with the Inaya Labs Auth Service at ${BASE_URL}. The tenant
key pair is in INAYA_API_KEY / INAYA_APP_KEY env vars (x-api-key /
x-client-app-key headers). Tenant ID: ${TENANT_ID}.
Steps: (1) POST /v3/auth/login_methods with {"locale":"en-US"} and report the
regional methods. (2) This is a QA environment: obtain a token pair via POST
/v3/auth/qa/login {"device_id":"agent-device","locale":"en-US","tenant_id":TENANT_ID,"username":"agent-user"}.
(3) Refresh it once via POST /v3/auth/refresh with the returned refresh_token
and the same tenant_id.
(4) Sign out via PUT /v3/auth/logout with Authorization: Bearer <token>.
Never log the tokens; report only success/failure per step.
```

## API reference

- [Auth Service — Swagger UI](https://auth-service-sandbox.inayalabs.com/v3/auth/swagger-ui/) · [Scalar](https://auth-service-sandbox.inayalabs.com/v3/auth/scalar)

## Next journeys

- **Calling Inaya services from an AI agent** (G-02) — agent-facing MCP access, no token handling at all.
- **Errors, retries & idempotency** (G-03) — what to do when these calls fail.

### G-02 · Calling services from an AI agent (MCP)
URL: /docs/guides/ai-agent-mcp.html

> Discover a service's agent-facing tools over MCP, list them, and call one — no API wrappers to write.

> **Journey.** As a developer of an AI agent, I want my agent to discover and call
> platform capabilities directly, so that I don't hand-write API integrations for
> every service.

Every Inaya Labs service that is agent-facing exposes an **MCP (Model Context
Protocol)** surface: a machine-readable tool catalog plus a calling convention.
An MCP-capable agent (or a script acting like one) connects to one URL per
service, asks what it can do, and calls it — no client SDK, no OpenAPI parsing at
runtime. This guide uses the Memcache Service as the running example; the same
pattern applies to every other service's `/agent/v1/mcp/*` endpoints.

## Prerequisites

- The Memcache Service is reachable (local stack `:40011`, or the shared dev
  subdomain).
- `curl`, or any HTTP client. Snippets also show Python and Node.
- **Sandbox exception.** In the Sandbox environment (`INAYA_LABS_ENVIRONMENT=sandbox`,
  the Playground stack) MCP endpoints accept any or no credentials and run under a
  synthetic `sandbox-user` identity in the `sandbox-tenant` tenant — tenant-argument
  tools (e.g. `add_cache_entry`) must pass `"tenant": "sandbox-tenant"` there.
  Outside the Sandbox, present the same tenant
  key-pair headers (`x-api-key` / `x-client-app-key`) on MCP endpoints as you would
  on REST endpoints, and pass your own tenant id.

## Concepts (only what this journey needs)

- **One endpoint per service.** `POST /agent/v1/mcp` (streamable HTTP) carries
  every JSON-RPC message and answers directly in its HTTP response — plain JSON
  when you send `Accept: application/json`, or a single SSE `message` event when
  you also accept `text/event-stream`. That one endpoint is the whole surface —
  the legacy SSE pair (`GET /agent/v1/mcp/sse` + `POST /agent/v1/mcp/message`)
  and the `initialize` handshake are not served (removed 2026-10-10).
- **JSON-RPC envelope.** Every message is `{"jsonrpc": "2.0", "id": <n>, "method":
  <name>, "params": {...}}`. Responses echo the `id` and carry either `result` or
  `error`. Keep `id` unique per in-flight request.
- **No session handshake (spec 2026-07-28).** Each request declares its protocol
  revision in `params._meta["io.modelcontextprotocol/protocolVersion"]` (or the
  `MCP-Protocol-Version` header, echoed on responses). Ask what the server speaks
  with `server/discover`; a revision the server does not support is rejected with
  `UnsupportedProtocolVersionError` (`-32022`) listing `data.supported`. The
  platform is latest-only: the legacy `initialize` handshake is not served.
- **Three methods cover everything**: `server/discover` (versions + capabilities),
  `tools/list` (tool catalog), `tools/call` (execution, with `params.name` and
  `params.arguments`).
- **Tools mirror the REST API.** Each MCP tool is a thin schema-guided wrapper over
  the service's own operation — identical behavior, gentler calling convention.
  Tools that touch tenant data take the tenant identity as an explicit argument
  (e.g. `tenant`), not from a header.

## Step 1 — Discover the server

Ask the service which protocol revisions and capabilities it supports. Modern
agents do this once, cache the answer (`ttlMs` permits it), and skip any
handshake:

```bash verify=auto
curl -s -X POST "${BASE_URL}/agent/v1/mcp" \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -d '{"jsonrpc": "2.0", "id": 1, "method": "server/discover", "params": {"_meta": {"io.modelcontextprotocol/protocolVersion": "2026-07-28", "io.modelcontextprotocol/clientInfo": {"name": "my-agent", "version": "1.0"}}}}'
```

```python verify=auto
import json, os, urllib.request

req = urllib.request.Request(
    f"{os.environ['BASE_URL']}/agent/v1/mcp",
    data=json.dumps({
        "jsonrpc": "2.0", "id": 1, "method": "server/discover",
        "params": {"_meta": {
            "io.modelcontextprotocol/protocolVersion": "2026-07-28",
            "io.modelcontextprotocol/clientInfo": {"name": "my-agent", "version": "1.0"},
        }},
    }).encode(),
    headers={"Content-Type": "application/json", "Accept": "application/json"},
)
print(json.dumps(json.load(urllib.request.urlopen(req)), indent=2))
```

```javascript verify=auto
const res = await fetch(`${process.env.BASE_URL}/agent/v1/mcp`, {
  method: "POST",
  headers: { "Content-Type": "application/json", Accept: "application/json" },
  body: JSON.stringify({
    jsonrpc: "2.0", id: 1, method: "server/discover",
    params: { _meta: {
      "io.modelcontextprotocol/protocolVersion": "2026-07-28",
      "io.modelcontextprotocol/clientInfo": { name: "my-agent", version: "1.0" },
    }},
  }),
});
console.log(await res.json());
```

**Expected response** — supported revisions, capabilities, and server identity:

```json
{ "jsonrpc": "2.0", "id": 1, "result": { "resultType": "complete", "supportedVersions": ["2026-07-28"], "capabilities": { "tools": {}, "resources": {} }, "_meta": { "io.modelcontextprotocol/serverInfo": { "name": "memcache-service", "version": "0.1.0" } }, "ttlMs": 3600000, "cacheScope": "public" } }
```

> [!NOTE]
> Every result also carries a `_meta["io.modelcontextprotocol/serverInfo"]`
> identity stamp; it is omitted from the remaining examples for brevity.

## Step 2 — Discover the tools

Ask the service what it can do. Agents should do this at session start rather
than hard-coding tool names. Each request carries its protocol revision in
`params._meta`:

```bash verify=auto
curl -s -X POST "${BASE_URL}/agent/v1/mcp" \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -d '{"jsonrpc": "2.0", "id": 2, "method": "tools/list", "params": {"_meta": {"io.modelcontextprotocol/protocolVersion": "2026-07-28"}}}'
```

```python verify=auto
import json, os, urllib.request

req = urllib.request.Request(
    f"{os.environ['BASE_URL']}/agent/v1/mcp",
    data=json.dumps({
        "jsonrpc": "2.0", "id": 2, "method": "tools/list",
        "params": {"_meta": {"io.modelcontextprotocol/protocolVersion": "2026-07-28"}},
    }).encode(),
    headers={"Content-Type": "application/json", "Accept": "application/json"},
)
print(json.dumps(json.load(urllib.request.urlopen(req)), indent=2))
```

```javascript verify=auto
const res = await fetch(`${process.env.BASE_URL}/agent/v1/mcp`, {
  method: "POST",
  headers: { "Content-Type": "application/json", Accept: "application/json" },
  body: JSON.stringify({
    jsonrpc: "2.0", id: 2, method: "tools/list",
    params: { _meta: { "io.modelcontextprotocol/protocolVersion": "2026-07-28" } },
  }),
});
console.log(await res.json());
```

**Expected response** — one entry per tool, each with a name, a plain-language
description, and a JSON-schema argument list:

```json
{
  "jsonrpc": "2.0",
  "id": 2,
  "result": {
    "tools": [
      { "name": "add_cache_entry", "description": "Stores a value in the in-memory cache for a tenant. …", "inputSchema": { "type": "object", "properties": { "tenant": { "type": "string" }, "key": { "type": "string" }, "value": { "type": "string" }, "ttl_in_seconds": { "type": "integer" } }, "required": ["tenant", "key", "value", "ttl_in_seconds"] } },
      { "name": "get_cache_entry", "description": "Retrieves and decrypts a value …", "inputSchema": { "type": "object", "properties": { "tenant": { "type": "string" }, "key": { "type": "string" } }, "required": ["tenant", "key"] } },
      { "name": "delete_cache_entry", "description": "Permanently removes an entry …", "inputSchema": { "type": "object", "properties": { "tenant": { "type": "string" }, "key": { "type": "string" } }, "required": ["tenant", "key"] } }
    ]
  }
}
```

## Step 3 — Call a tool

Store a value, then read it back. Note that `tenant` is an explicit argument and
must match the caller's tenant claims — `sandbox-tenant` in the Sandbox
environment, your own tenant id elsewhere:

```bash verify=auto
curl -s -X POST "${BASE_URL}/agent/v1/mcp" \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -d '{"jsonrpc": "2.0", "id": 3, "method": "tools/call", "params": {"name": "add_cache_entry", "arguments": {"tenant": "sandbox-tenant", "key": "agent-demo", "value": "hello from the agent", "ttl_in_seconds": 300}, "_meta": {"io.modelcontextprotocol/protocolVersion": "2026-07-28"}}}'
```

```python verify=auto
import json, os, urllib.request

def call(base, msg_id, method, params):
    params = {**params, "_meta": {"io.modelcontextprotocol/protocolVersion": "2026-07-28"}}
    req = urllib.request.Request(
        f"{base}/agent/v1/mcp",
        data=json.dumps({"jsonrpc": "2.0", "id": msg_id,
                         "method": method, "params": params}).encode(),
        headers={"Content-Type": "application/json", "Accept": "application/json"},
    )
    return json.load(urllib.request.urlopen(req))

base = os.environ["BASE_URL"]
print(json.dumps(call(base, 3, "tools/call", {
    "name": "add_cache_entry",
    "arguments": {"tenant": "sandbox-tenant", "key": "agent-demo",
                  "value": "hello from the agent", "ttl_in_seconds": 300},
}), indent=2))
```

```javascript verify=auto
const call = async (id, method, params) => {
  const res = await fetch(`${process.env.BASE_URL}/agent/v1/mcp`, {
    method: "POST",
    headers: { "Content-Type": "application/json", Accept: "application/json" },
    body: JSON.stringify({ jsonrpc: "2.0", id, method,
      params: { ...params, _meta: { "io.modelcontextprotocol/protocolVersion": "2026-07-28" } } }),
  });
  return res.json();
};
console.log(await call(3, "tools/call", {
  name: "add_cache_entry",
  arguments: { tenant: "sandbox-tenant", key: "agent-demo",
               value: "hello from the agent", ttl_in_seconds: 300 },
}));
```

**Expected response** — tool results arrive as content items; structured output is
embedded JSON text. (`isError` appears only when the call fails, so its absence
means success.)

```json
{ "jsonrpc": "2.0", "id": 3, "result": { "content": [{ "type": "text", "text": "{\"stored\":true,\"tenant\":\"sandbox-tenant\",\"key\":\"agent-demo\"}" }] } }
```

Read it back with `get_cache_entry` (same `tenant` and `key`):

```bash verify=auto
curl -s -X POST "${BASE_URL}/agent/v1/mcp" \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -d '{"jsonrpc": "2.0", "id": 4, "method": "tools/call", "params": {"name": "get_cache_entry", "arguments": {"tenant": "sandbox-tenant", "key": "agent-demo"}, "_meta": {"io.modelcontextprotocol/protocolVersion": "2026-07-28"}}}'
```

**Expected response**:

```json
{ "jsonrpc": "2.0", "id": 4, "result": { "content": [{ "type": "text", "text": "{\"found\":true,\"value\":\"hello from the agent\"}" }] } }
```

> [!PITFALL]
> A *miss* is not an error: `get_cache_entry` returns `{"found": false}` in a
> perfectly successful result for a missing or expired key. Agents should branch
> on `found`, not on the presence of `isError`. Also respect `ttl_in_seconds` —
> an entry that silently disappears was expired, not deleted.

> [!TIP]
> Clean up after the demo: call `delete_cache_entry` with the same `tenant` and
> `key` when your agent is done — cache space is shared per tenant.

## Full working example

A complete discover-then-call cycle — save as `agent_probe.sh`:

```bash verify=auto
#!/usr/bin/env bash
set -euo pipefail

MCP="${BASE_URL}/agent/v1/mcp"
META='"_meta": {"io.modelcontextprotocol/protocolVersion": "2026-07-28"}'
HEADERS=(-H "Content-Type: application/json" -H "Accept: application/json")

echo "1. Discover:"
curl -sf -X POST "$MCP" "${HEADERS[@]}" \
  -d "{\"jsonrpc\": \"2.0\", \"id\": 1, \"method\": \"server/discover\", \"params\": {$META}}" \
  | python3 -c 'import json,sys; d=json.load(sys.stdin); print("  ", ", ".join(d["result"]["supportedVersions"]))'

echo "2. Tools:"
curl -sf -X POST "$MCP" "${HEADERS[@]}" \
  -d "{\"jsonrpc\": \"2.0\", \"id\": 2, \"method\": \"tools/list\", \"params\": {$META}}" \
  | python3 -c 'import json,sys; d=json.load(sys.stdin); print("  ", ", ".join(t["name"] for t in d["result"]["tools"]))'

echo "3. Store:"
curl -sf -X POST "$MCP" "${HEADERS[@]}" \
  -d "{\"jsonrpc\": \"2.0\", \"id\": 3, \"method\": \"tools/call\", \"params\": {\"name\": \"add_cache_entry\", \"arguments\": {\"tenant\": \"sandbox-tenant\", \"key\": \"agent-demo\", \"value\": \"round-trip\", \"ttl_in_seconds\": 300}, $META}}" > /dev/null \
  && echo "   ok"

echo "4. Retrieve:"
curl -sf -X POST "$MCP" "${HEADERS[@]}" \
  -d "{\"jsonrpc\": \"2.0\", \"id\": 4, \"method\": \"tools/call\", \"params\": {\"name\": \"get_cache_entry\", \"arguments\": {\"tenant\": \"sandbox-tenant\", \"key\": \"agent-demo\"}, $META}}"
echo

echo "5. Cleanup:"
curl -sf -X POST "$MCP" "${HEADERS[@]}" \
  -d "{\"jsonrpc\": \"2.0\", \"id\": 5, \"method\": \"tools/call\", \"params\": {\"name\": \"delete_cache_entry\", \"arguments\": {\"tenant\": \"sandbox-tenant\", \"key\": \"agent-demo\"}, $META}}" > /dev/null \
  && echo "   ok"
```

## Try it live

The same Memcache Service backs the [Playground](../../playgrounds/playground.html)
→ **Infrastructure** → *Memcache* console. Run the snippets above against the same
base URL the Playground uses to watch both surfaces hit one service.

## Copy agent prompt

```prompt
You can use the Inaya Labs Memcache Service via MCP at ${BASE_URL}
(e.g. http://127.0.0.1:40011). Every request is a JSON-RPC POST to
/agent/v1/mcp with headers "Content-Type: application/json" and
"Accept: application/json", and carries
params._meta["io.modelcontextprotocol/protocolVersion"] = "2026-07-28".
Steps: (1) Call server/discover with id 1 and report the supported protocol
versions. (2) Call tools/list with id 2 and report the available tools. (3) Call
tools/call for add_cache_entry with {"tenant":"sandbox-tenant","key":"agent-prompt-demo","value":"...","ttl_in_seconds":300},
then get_cache_entry to read it back, then delete_cache_entry to clean up.
(4) Branch on the "found" field for reads, not on isError. Report each step's
outcome briefly.
```

## API reference

- [Memcache Service — Swagger UI](https://memcache-service-sandbox.inayalabs.com/v3/mem-cache/swagger-ui/) · [Scalar](https://memcache-service-sandbox.inayalabs.com/v3/mem-cache/scalar)

## Next journeys

- **Uploading files the platform way** (G-04) — when your agent handles files, use the CDN's REST upload rather than MCP.
- **Errors, retries & idempotency** (G-03) — how MCP `error` results and HTTP statuses map to retry decisions.

### G-03 · Errors, retries & idempotency
URL: /docs/guides/errors-retries-idempotency.html

> Read platform error statuses correctly, retry only what's retryable, and design writes so retries are safe.

> **Journey.** As a developer, I want my integration to fail predictably, retry
> only transient failures, and never corrupt state when a call is repeated, so
> that my app survives the messy reality of networks and services.

Production integrations live or die by what they do when things go wrong. This
guide shows the platform's failure vocabulary, a safe retry pattern, and how to
make repeated writes harmless. It uses the Memcache Service (`:40011`) in the
**local** environment — Sandbox accepts everything, which defeats the purpose.

## Prerequisites

- The Memcache Service is reachable in a non-Sandbox environment (local stack
  `:40011`) — error statuses below assume real authentication is enforced.
- Your tenant key pair in `INAYA_API_KEY` / `INAYA_APP_KEY`, **and** a user JWT in
  `TOKEN` — Memcache endpoints require both (obtain one via guide G-01; the full
  example below fetches one via the QA sign-in shortcut).
- `curl`, or any HTTP client. Snippets also show Python and Node.

## Concepts (only what this journey needs)

- **Two credential tiers fail separately.** A missing or rejected key pair *or* a
  missing or expired JWT both produce `401` with an `error` message telling you
  which — `{"error":"Missing Authorization token"}` vs a key-pair rejection.
  Knowing which credential failed tells you whether to fix the pair (G-00) or
  re-authenticate (G-01).

- **The `error` envelope.** Failure bodies are plain text or a JSON string:
  `{"error": "…"}` — read the message before deciding what to do.
- **`4xx` means *you*; `5xx`/`000` means *them*.** `4xx` responses will fail
  identically on retry — fix the request. `5xx` and connection failures
  (`000`, timeouts) are the only statuses worth retrying.
- **Retry with backoff + jitter, and cap it.** Two to four attempts, growing
  waits, random jitter, and a hard ceiling. Then surface the failure.
- **Idempotency is a design choice, not a header.** The platform does not issue
  idempotency keys. Safe retries come from *your* identifiers: deterministic
  keys, upsert/overwrite semantics, or natural uniqueness in the payload.

## Step 1 — Learn the failure vocabulary

Probe three representative failures and read the status line:

```bash verify=auto
echo "1. Missing key pair and JWT (expected 401):"
curl -s -o /tmp/last_error.json -w "   HTTP %{http_code}\n" \
  -X POST "${BASE_URL}/v3/mem-cache/add/demo-tenant" \
  -H "Content-Type: application/json" \
  -d '{"key": "k", "value": "v", "ttl_in_seconds": 60}'
cat /tmp/last_error.json; echo

echo "2. Unknown path, fully authenticated (expected 404):"
curl -s -o /dev/null -w "   HTTP %{http_code}\n" \
  "${BASE_URL}/v3/mem-cache/no-such-endpoint" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}"

echo "3. Malformed body, fully authenticated (expected 422):"
curl -s -o /dev/null -w "   HTTP %{http_code}\n" \
  -X POST "${BASE_URL}/v3/mem-cache/add/demo-tenant" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d '{"key": "k"}'
```

```python verify=auto
import json, os, urllib.request, urllib.error

base = os.environ["BASE_URL"]
headers = {
    "Content-Type": "application/json",
    "Authorization": f"Bearer {os.environ['TOKEN']}",
    "x-api-key": os.environ["INAYA_API_KEY"],
    "x-client-app-key": os.environ["INAYA_APP_KEY"],
}

def probe(label, req, expected):
    try:
        code = urllib.request.urlopen(req).status
    except urllib.error.HTTPError as e:
        code = e.code
    print(f"{label}: HTTP {code} (expected {expected})")

probe("1. Missing key pair and JWT",
      urllib.request.Request(f"{base}/v3/mem-cache/add/demo-tenant",
          data=json.dumps({"key": "k", "value": "v", "ttl_in_seconds": 60}).encode(),
          headers={"Content-Type": "application/json"}), 401)
probe("2. Unknown path",
      urllib.request.Request(f"{base}/v3/mem-cache/no-such-endpoint",
          headers=headers), 404)
probe("3. Malformed body",
      urllib.request.Request(f"{base}/v3/mem-cache/add/demo-tenant",
          data=json.dumps({"key": "k"}).encode(), headers=headers), 422)
```

```javascript verify=auto
const base = process.env.BASE_URL;
const headers = {
  "Content-Type": "application/json",
  Authorization: `Bearer ${process.env.TOKEN}`,
  "x-api-key": process.env.INAYA_API_KEY,
  "x-client-app-key": process.env.INAYA_APP_KEY,
};

const probe = async (label, input, expected) => {
  const res = await fetch(`${base}${input.path}`, input).catch(() => null);
  console.log(`${label}: HTTP ${res ? res.status : "000"} (expected ${expected})`);
};

await probe("1. Missing key pair and JWT", {
  path: "/v3/mem-cache/add/demo-tenant", method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({ key: "k", value: "v", ttl_in_seconds: 60 }),
}, 401);
await probe("2. Unknown path", { path: "/v3/mem-cache/no-such-endpoint", headers }, 404);
await probe("3. Malformed body", {
  path: "/v3/mem-cache/add/demo-tenant", method: "POST",
  headers, body: JSON.stringify({ key: "k" }),
}, 422);
```

**Expected output** — probe 1's body names the missing credential; the rest are
status codes:

```
1. Missing key pair and JWT (expected 401):
   HTTP 401
{"error": "Missing Authorization token"}
2. Unknown path, fully authenticated (expected 404):
   HTTP 404
3. Malformed body, fully authenticated (expected 422):
   HTTP 422
```

The mapping generalizes across every platform service:

| Status | Meaning | Correct action |
|---|---|---|
| `400` | Business-rule rejection — the request was understood but refused | Read the message; change the request or treat as terminal |
| `401` | Key pair rejected, or missing JWT | Re-check credentials; re-authenticate via G-01 if token-based |
| `403` | Authenticated but not allowed (e.g. rotated refresh token) | Terminal — do not retry; re-authenticate |
| `404` | Wrong path or unknown resource id | Fix the URL/id; terminal |
| `409` | Conflict, e.g. CDN file exists without `overwrite=true` | Resolve the conflict deliberately (see G-04) |
| `422` | Body failed schema validation | Fix the payload; terminal |
| `429` | Rate limited | Back off *longer* than usual, then retry |
| `5xx` / `000` | Service or network failure | Retry with backoff (Step 2) |

> [!PITFALL]
> **Never retry `4xx`.** A `422` retried ten times is ten guaranteed failures
> plus self-inflicted load. Retry logic must branch on the status class first.

## Step 2 — Retry only what's retryable

A bounded retry helper: exponential backoff, jitter, and a small attempt cap.
It retries connection failures and `5xx`/`429` only:

```bash verify=auto
#!/usr/bin/env bash
set -uo pipefail

call_with_retry() {
  local attempt=1 max=4 status
  while true; do
    status=$(curl -s -o /tmp/g03_last.json -w "%{http_code}" "$@")
    if [[ "$status" -lt 500 && "$status" -ne 000 && "$status" -ne 429 ]]; then
      echo "$status"; return 0
    fi
    if [[ "$attempt" -ge "$max" ]]; then
      echo "$status"; return 1
    fi
    sleep "$(( (2 ** attempt) + RANDOM % 2 ))"
    attempt=$((attempt + 1))
  done
}

code=$(call_with_retry -X GET "${BASE_URL}/v3/mem-cache/get/demo-tenant/some-key" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}")
echo "final status: $code"
cat /tmp/g03_last.json; echo
```

```python verify=auto
import json, os, random, time, urllib.request, urllib.error

def get_with_retry(path, attempts=4):
    url = f"{os.environ['BASE_URL']}{path}"
    headers = {
        "Authorization": f"Bearer {os.environ['TOKEN']}",
        "x-api-key": os.environ["INAYA_API_KEY"],
        "x-client-app-key": os.environ["INAYA_APP_KEY"],
    }
    for attempt in range(1, attempts + 1):
        try:
            req = urllib.request.Request(url, headers=headers)
            res = urllib.request.urlopen(req)
            return res.status, json.load(res)
        except urllib.error.HTTPError as e:
            if (e.code < 500 and e.code != 429) or attempt == attempts:
                return e.code, None
        except urllib.error.URLError:
            if attempt == attempts:
                return 0, None
        time.sleep(2 ** attempt + random.random())

status, _ = get_with_retry("/v3/mem-cache/get/demo-tenant/some-key")
print("final status:", status)
```

```javascript verify=auto
const getWithRetry = async (path, attempts = 4) => {
  for (let attempt = 1; ; attempt++) {
    try {
      const res = await fetch(`${process.env.BASE_URL}${path}`, {
        headers: {
          Authorization: `Bearer ${process.env.TOKEN}`,
          "x-api-key": process.env.INAYA_API_KEY,
          "x-client-app-key": process.env.INAYA_APP_KEY,
        },
      });
      if ((res.status < 500 && res.status !== 429) || attempt === attempts)
        return res.status;
    } catch {
      if (attempt === attempts) return 0;
    }
    await new Promise(r => setTimeout(r, 2 ** attempt * 1000 + Math.random() * 1000));
  }
};
console.log("final status:", await getWithRetry("/v3/mem-cache/get/demo-tenant/some-key"));
```

**Expected output** — succeeds on the first attempt against a healthy service.
A cache miss on this REST endpoint is a `404` with an empty body — a terminal,
non-retryable status, exactly what the helper is designed to stop on. (The MCP
tool for the same service reports misses as `{"found": false}` instead — see
G-02; conventions differ across surfaces, so always check the docs for the
surface you use.)

```
final status: 404
```

> [!NOTE]
> **Design for parallel safety.** If several of your instances can issue the
> same write, serialize retries per key — or make the write idempotent (Step 3)
> so racing retries converge instead of colliding.

## Step 3 — Make retries harmless: idempotency by design

The platform has no idempotency-key header. Instead, choose identifiers and
semantics that make repetition safe:

- **Deterministic keys + upsert semantics.** Memcache `add` *overwrites* an
  existing key (last write wins) and returns `201` whether or not the key
  existed. Derive the key from the business object (`job-42-snapshot`), not a
  random UUID: a retried store lands on the same key with the same value, and
  the state converges by construction. (Where a service offers an explicit
  overwrite switch instead — the CDN's `overwrite=true`, G-04 — use it when
  re-sending is expected.)
- **Read-modify-write with natural keys.** `GET` then re-`POST` flows keyed by
  a natural identifier survive repetition because the final state equals the
  last write.

Demonstrate convergence: two identical stores under a deterministic key both
succeed, and the read-back shows a single entry with the expected value:

```bash verify=auto
#!/usr/bin/env bash
set -uo pipefail
HEADERS=(-H "Content-Type: application/json"
         -H "Authorization: Bearer ${TOKEN}"
         -H "x-api-key: ${INAYA_API_KEY}"
         -H "x-client-app-key: ${INAYA_APP_KEY}")
BODY='{"key": "job-42-snapshot", "value": "{\"status\": \"done\"}", "ttl_in_seconds": 300}'

# First store: 201
code1=$(curl -s -o /tmp/g03_add1.json -w "%{http_code}" -X POST \
  "${BASE_URL}/v3/mem-cache/add/demo-tenant" "${HEADERS[@]}" -d "$BODY")
# Retry (e.g. a client that never saw the response): 201 again — upsert
code2=$(curl -s -o /tmp/g03_add2.json -w "%{http_code}" -X POST \
  "${BASE_URL}/v3/mem-cache/add/demo-tenant" "${HEADERS[@]}" -d "$BODY")
# State check: the GET returns the stored value directly
final=$(curl -s "${BASE_URL}/v3/mem-cache/get/demo-tenant/job-42-snapshot" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" -H "x-client-app-key: ${INAYA_APP_KEY}")

echo "first: $code1, retry: $code2 (upsert — both succeed)"
echo "final state: $final"
```

**Expected output**:

```
first: 201, retry: 201 (upsert — both succeed)
final state: {"status": "done"}
```

> [!PITFALL]
> **Random IDs make retries dangerous.** If your client generates a fresh UUID
> per attempt, a retried store creates *orphan* entries instead of converging.
> Generate business IDs once, at the source, and reuse them across retries.

## Full working example

A resilient store helper — deterministic key, duplicate-tolerant, retried only
on transient failures (`retry_safe_store.sh`). It first fetches a user JWT via
the QA sign-in shortcut (local/QA environments only; in production, obtain the
token through the passkey/federated flow from G-01):

```bash verify=auto
#!/usr/bin/env bash
set -euo pipefail

TENANT="demo-tenant"
KEY="job-42-snapshot"

# 0. Obtain a user JWT (QA shortcut — see G-01 for production flows)
TOKEN=$(curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"device_id\": \"retry-demo\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"retry-demo\"}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])')

HEADERS=(-H "Content-Type: application/json"
         -H "Authorization: Bearer ${TOKEN}"
         -H "x-api-key: ${INAYA_API_KEY}"
         -H "x-client-app-key: ${INAYA_APP_KEY}")

# 1. Store (upsert — safe to repeat)
code=$(curl -s -o /tmp/g03_full_last.json -w "%{http_code}" -X POST \
  "${BASE_URL}/v3/mem-cache/add/${TENANT}" "${HEADERS[@]}" \
  -d "{\"key\": \"${KEY}\", \"value\": \"done\", \"ttl_in_seconds\": 300}")
if [[ "$code" != "201" ]]; then
  echo "unexpected: $code"; cat /tmp/g03_full_last.json; exit 1
fi
echo "stored"

# 2. Read back
curl -sf "${BASE_URL}/v3/mem-cache/get/${TENANT}/${KEY}" "${HEADERS[@]}"
echo
```

(`AUTH_URL` is the Auth Service base URL — `http://127.0.0.1:40004` on the local
stack; `TENANT_ID` is your tenant's id.)

## Try it live

Open the [Playground](../../playgrounds/playground.html) → **Infrastructure** →
*Memcache* console and repeat the same store twice — both attempts succeed, and
the second simply overwrites: the upsert convergence from Step 3, live.

## Copy agent prompt

```prompt
You are hardening an integration with the Inaya Labs Memcache Service at
${BASE_URL}. The tenant key pair is in INAYA_API_KEY / INAYA_APP_KEY env vars,
and a user JWT is in TOKEN (if empty, obtain one via the QA login described in
guide G-01, then send it as Authorization: Bearer on every call). Rules:
(1) Never retry 4xx statuses — report them as terminal with the error message.
(2) Retry only 5xx/429/connection failures, max 4 attempts, exponential
backoff 2s→4s→8s with jitter. (3) All keys must be deterministic: derive them
from the business object, never generate fresh UUIDs per attempt. (4) Memcache
add is an upsert (201, last write wins) — a repeated identical store converges;
verify with GET /v3/mem-cache/get/{tenant}/{key}, which returns the raw stored
value (404 with an empty body when missing). Demonstrate by storing under key
"job-42-snapshot" for tenant "demo-tenant", "retrying" the same store, and
reading the value back.
```

## API reference

- [Memcache Service — Swagger UI](https://memcache-service-sandbox.inayalabs.com/v3/mem-cache/swagger-ui/) · [Scalar](https://memcache-service-sandbox.inayalabs.com/v3/mem-cache/scalar)

## Next journeys

- **Uploading files the platform way** (G-04) — the `409`/`overwrite` conflict pattern in action.
- **Authentication & sessions** (G-01) — how to re-authenticate when `401`/`403` do occur.

### G-04 · Uploading files the platform way
URL: /docs/guides/file-upload.html

> Upload a file through the CDN, handle the already-exists conflict with overwrite, read it back, delegate access with a time-limited token, and delete it.

> **Journey.** As a developer, I want my app to store files (avatars, voice
> recordings, documents), share them safely, and clean them up, without
> building storage infrastructure.

The CDN Service is the platform's single front door for files. Every other
service that consumes files — Speech, Media Editing, RAG ingestion — takes a
*path* that refers to a file stored here. This guide covers the full lifecycle:
upload, the already-exists conflict, read-back, delegation via time-limited
tokens, and deletion. Verified against the local stack (CDN `:40040`); the
shared dev subdomain works identically.

## Prerequisites

- The CDN Service is reachable (local stack `:40040`, or the shared dev
  subdomain).
- Your tenant key pair in `INAYA_API_KEY` / `INAYA_APP_KEY`, **and** a user JWT
  in `TOKEN` — CDN file endpoints require both (obtain one via guide G-01; the
  full example below fetches one via the QA sign-in shortcut).
- `curl`, or any HTTP client. Snippets also show Python (uses `requests`) and
  Node (built-in `fetch`/`FormData`, Node ≥ 18).

## Concepts (only what this journey needs)

- **Paths, not URLs.** You store and reference files by a tenant-relative path
  (`uploads/avatar.png`). The service manages the physical location; other
  services consume the same path via their `source_path`-style fields.
- **Ownership follows the uploader.** Files are recorded against the uploading
  user's id (`sub`). Only the owner (or an admin role) can download or delete
  them via the authenticated endpoints.
- **Conflicts are explicit.** Re-uploading an existing path without
  `overwrite=true` fails with `409` — the platform never silently replaces
  content. With the flag, the replace is atomic.
- **Sharing uses scoped tokens.** To hand a file to another component of your
  system, mint a time-limited token: it binds the download to one path and
  expiry. The caller still presents the usual key pair and a valid JWT — the
  token scopes *which* file the call may stream.

## Step 1 — Upload a file

Uploads are `multipart/form-data` with two fields: `path` (the tenant-relative
destination) and `file` (the bytes):

```bash verify=auto
curl -s -o /dev/null -w "HTTP %{http_code}\n" \
  -X POST "${BASE_URL}/v3/cdn/files/upload" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -F "path=demo/hello.txt" \
  -F "file=@./hello.txt;type=text/plain"
```

```python verify=auto
import os, requests

res = requests.post(
    f"{os.environ['BASE_URL']}/v3/cdn/files/upload",
    headers={
        "Authorization": f"Bearer {os.environ['TOKEN']}",
        "x-api-key": os.environ["INAYA_API_KEY"],
        "x-client-app-key": os.environ["INAYA_APP_KEY"],
    },
    data={"path": "demo/hello.txt"},
    files={"file": ("hello.txt", open("./hello.txt", "rb"), "text/plain")},
)
print("HTTP", res.status_code)
```

```javascript verify=auto
import { readFileSync } from "node:fs";

const form = new FormData();
form.set("path", "demo/hello.txt");
form.set("file", new Blob([readFileSync("./hello.txt")], { type: "text/plain" }));

const res = await fetch(`${process.env.BASE_URL}/v3/cdn/files/upload`, {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.TOKEN}`,
    "x-api-key": process.env.INAYA_API_KEY,
    "x-client-app-key": process.env.INAYA_APP_KEY,
  },
  body: form,
});
console.log("HTTP", res.status);
```

**Expected response** — success is a bare `201` (empty body):

```
HTTP 201
```

> [!PITFALL]
> Multipart field order does not matter, but the field *names* do — `path` and
> `file` exactly. A JSON body here returns `400`, and a `415` means you sent
> the wrong `Content-Type`: let your HTTP client generate the multipart
> boundary, never set `Content-Type` manually.

## Step 2 — The already-exists conflict, and overwrite

Upload the same path again and the platform refuses to guess:

```bash verify=auto
echo "1. Same path, no overwrite (expected 409):"
curl -s -o /tmp/g04_conflict.json -w "   HTTP %{http_code}\n" \
  -X POST "${BASE_URL}/v3/cdn/files/upload" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -F "path=demo/hello.txt" \
  -F "file=@./hello-v2.txt;type=text/plain"
cat /tmp/g04_conflict.json; echo

echo "2. Same path, overwrite=true (expected 201):"
curl -s -o /dev/null -w "   HTTP %{http_code}\n" \
  -X POST "${BASE_URL}/v3/cdn/files/upload?overwrite=true" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -F "path=demo/hello.txt" \
  -F "file=@./hello-v2.txt;type=text/plain"
```

**Expected output**:

```
1. Same path, no overwrite (expected 409):
   HTTP 409
File already exists; set overwrite=true to replace it
2. Same path, overwrite=true (expected 201):
   HTTP 201
```

This is the `409` pattern from G-03: deliberate, never silent. Use
`overwrite=true` when re-sending is expected (re-exports, regenerated
documents); surface the conflict to the user when overwriting would destroy
something a human chose.

## Step 3 — Read the file back

The authenticated download requires the owner's JWT — other users get `403`:

```bash verify=auto
curl -s -o /tmp/hello-back.txt -w "HTTP %{http_code}\n" \
  "${BASE_URL}/v3/cdn/files/private/download?path=demo/hello.txt" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}"
diff ./hello-v2.txt /tmp/hello-back.txt && echo "content matches"
```

**Expected output**:

```
HTTP 200
content matches
```

## Step 4 — Delegate access with a time-limited token

For browsers, third parties, or other services, mint a token that carries its
own expiry and signature:

```bash verify=auto
TOKEN_RESP=$(curl -s -X POST "${BASE_URL}/v3/cdn/files/token" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d '{"path": "demo/hello.txt"}')
echo "$TOKEN_RESP"
DL_TOKEN=$(echo "$TOKEN_RESP" | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])')

# The token scopes the download to this file; caller credentials are still required
curl -s -o /tmp/hello-token.txt -w "HTTP %{http_code}\n" \
  "${BASE_URL}/v3/cdn/files/download?token=${DL_TOKEN}" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}"
diff ./hello-v2.txt /tmp/hello-token.txt && echo "content matches"
```

**Expected output** — the token response carries the expiry as a Unix
timestamp, and the tokenized download streams the file when the caller presents
the usual credentials:

```json
{ "token": "…", "expires_at": 1790205431 }
```

```
HTTP 200
content matches
```

> [!NOTE]
> The default TTL is configurable per deployment (`INAYA_LABS_CDN_TOKEN_TTL_IN_SECONDS`).
> The token scopes the download to one file until it expires — but caller
> credentials (key pair + JWT) are still required, so treat it as a capability
> on top of identity, not an anonymous link. Issue short TTLs for anything
> sensitive.

## Step 5 — Delete the file

Cleanup uses the path itself (`DELETE`, wildcard route):

```bash verify=auto
curl -s -o /dev/null -w "HTTP %{http_code}\n" -X DELETE \
  "${BASE_URL}/v3/cdn/files/demo/hello.txt" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}"
```

**Expected response** — `204`, and a later authenticated read of the same path
returns `403` ("no ownership record") — deletion removes both the bytes and
the ownership record.

## Full working example

A complete upload → conflict → overwrite → download → delete cycle
(`cdn_lifecycle.sh`) — it fetches a user JWT via the QA shortcut first
(local/QA environments only; production uses the G-01 flows):

```bash verify=auto
#!/usr/bin/env bash
set -euo pipefail

printf 'hello from the CDN guide\n' > ./hello.txt
printf 'hello v2\n' > ./hello-v2.txt

# 0. Obtain a user JWT (QA shortcut — see G-01 for production flows)
TOKEN=$(curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"device_id\": \"cdn-demo-device\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"cdn-demo\"}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])')

AUTH=(-H "Authorization: Bearer ${TOKEN}"
      -H "x-api-key: ${INAYA_API_KEY}"
      -H "x-client-app-key: ${INAYA_APP_KEY}")

# 1. Upload
curl -sf -X POST "${BASE_URL}/v3/cdn/files/upload" "${AUTH[@]}" \
  -F "path=demo/hello.txt" -F "file=@./hello.txt;type=text/plain" \
  && echo "uploaded"

# 2. Conflict, then overwrite
code=$(curl -s -o /dev/null -w "%{http_code}" -X POST \
  "${BASE_URL}/v3/cdn/files/upload" "${AUTH[@]}" \
  -F "path=demo/hello.txt" -F "file=@./hello-v2.txt;type=text/plain")
[[ "$code" == "409" ]] && echo "conflict surfaced: 409"
curl -sf -X POST "${BASE_URL}/v3/cdn/files/upload?overwrite=true" "${AUTH[@]}" \
  -F "path=demo/hello.txt" -F "file=@./hello-v2.txt;type=text/plain" \
  && echo "overwritten"

# 3. Read back
curl -sf "${BASE_URL}/v3/cdn/files/private/download?path=demo/hello.txt" \
  "${AUTH[@]}" -o /tmp/hello-back.txt && diff ./hello-v2.txt /tmp/hello-back.txt \
  && echo "read-back matches"

# 4. Delete
curl -sf -X DELETE "${BASE_URL}/v3/cdn/files/demo/hello.txt" "${AUTH[@]}" \
  && echo "deleted"
```

(`AUTH_URL` is the Auth Service base URL — `http://127.0.0.1:40004` on the local
stack; `TENANT_ID` is your tenant's id.)

## Try it live

Open the [Playground](../../playgrounds/playground.html) → **Infrastructure** →
*CDN* console — it exercises the same upload/download/delete cycle against the
live CDN Service with a file picker.

## Copy agent prompt

```prompt
You are working with the Inaya Labs CDN Service at ${BASE_URL}. The tenant key
pair is in INAYA_API_KEY / INAYA_APP_KEY env vars; obtain a user JWT via the QA
login from guide G-01 if TOKEN is not already set. Steps: (1) Upload
./hello.txt to path "demo/hello.txt" via POST /v3/cdn/files/upload (multipart
fields "path" and "file"); expect 201. (2) Upload again unchanged and report
the 409; then re-upload with ?overwrite=true and expect 201. (3) Read back via
GET /v3/cdn/files/private/download?path=demo/hello.txt and confirm the content
round-trips. (4) Mint a download token via POST /v3/cdn/files/token {"path":
"demo/hello.txt"} and verify GET /v3/cdn/files/download?token=<token> streams
the file when called with your key pair and JWT (the token scopes which file).
(5) Delete via DELETE /v3/cdn/files/demo/hello.txt; expect 204. Never log the
JWT; report status codes per step.
```

## API reference

- [CDN Service — Swagger UI](https://cdn-service-sandbox.inayalabs.com/v3/cdn/swagger-ui/) · [Scalar](https://cdn-service-sandbox.inayalabs.com/v3/cdn/scalar)

## Next journeys

- **Errors, retries & idempotency** (G-03) — the retry discipline that makes uploads safe to repeat.
- **Calling services from an AI agent** (G-02) — the CDN's MCP tools include token generation for agent-driven sharing.

### G-10 · Your first AI chat session
URL: /docs/guides/first-ai-chat-session.html

> Create an Evergreen AI session with model-selection criteria, send a message, stream the reply, and end the session cleanly.

> **Journey.** As a developer, I want to open a conversation with an AI model I
> choose by capability, send messages, and get answers — without hosting or
> managing any model infrastructure.

Evergreen AI is the platform's model-front-door service: you describe *what you
need* (criteria), it selects and loads a model, and you exchange messages over a
session. This guide walks the full lifecycle — create, chat, stream, review
history, end — against the local stack (Evergreen AI `:40071`). Verified with a
locally-running Ollama backend.

## Prerequisites

- The Evergreen AI Service is reachable (local stack `:40071`, or the shared
  dev subdomain).
- Your tenant key pair in `INAYA_API_KEY` / `INAYA_APP_KEY`, **and** a user JWT
  in `TOKEN` (obtain one via guide G-01; the full example below fetches one via
  the QA sign-in shortcut).
- An inference backend with at least one model available. On the local stack
  this is Ollama (`http://127.0.0.1:11434`) with a model pulled, e.g.
  `qwen3-vl:4b`.
- `curl`, or any HTTP client. Snippets also show Python and Node.
- **Patience for the first call.** The first message in a session pays the
  model-load cost (tens of seconds on a laptop); later messages in the same
  session are fast. On a memory-constrained dev machine the first call can
  stretch to several minutes — size your client timeouts accordingly.

## Concepts (only what this journey needs)

- **A session is a configured conversation.** Creating one picks the model,
  records generation parameters (temperature, system prompt, …), and allocates
  server-side history. Sessions expire on their own (`history_ttl_in_seconds`,
  default 24 h) — treat the id as ephemeral.
- **You select models by criteria, not by URL.** `selection_criteria` is a
  filter list (specific model, architecture, parameter count, minimum context
  window, capability, …) each marked `MustHave` or `NiceToHave`. The service
  picks the best locally-available match. This guide pins one model
  (`MustHave` + `Model`) for determinism.
- **Two ways to receive a reply.** `POST …/chat-messages` returns the complete
  `AiChatMessage` JSON; `POST …/streamed-chat-messages` returns SSE `message`
  events carrying text *deltas* you concatenate, terminated by `done`.
- **History is server-side.** Every exchange is recorded against the session;
  `GET …/chat-history` replays it. Ending the session (`DELETE`) discards it.

## Step 1 — Create a session

```bash verify=auto
curl -s -X POST "${BASE_URL}/v3/sessions" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d '{
    "required_modalities": ["Text"],
    "selection_criteria": [
      { "Model": { "optionality": "MustHave", "name": "Qwen3_VL_4B_Q4" } }
    ],
    "system_prompt": "You are a helpful assistant. Answer in one short sentence.",
    "temperature": 0.2
  }'
```

```python verify=auto
import json, os, urllib.request

req = urllib.request.Request(
    f"{os.environ['BASE_URL']}/v3/sessions",
    data=json.dumps({
        "required_modalities": ["Text"],
        "selection_criteria": [
            {"Model": {"optionality": "MustHave", "name": "Qwen3_VL_4B_Q4"}},
        ],
        "system_prompt": "You are a helpful assistant. Answer in one short sentence.",
        "temperature": 0.2,
    }).encode(),
    headers={
        "Content-Type": "application/json",
        "Authorization": f"Bearer {os.environ['TOKEN']}",
        "x-api-key": os.environ["INAYA_API_KEY"],
        "x-client-app-key": os.environ["INAYA_APP_KEY"],
    },
)
print(json.dumps(json.load(urllib.request.urlopen(req, timeout=60)), indent=2))
```

```javascript verify=auto
const res = await fetch(`${process.env.BASE_URL}/v3/sessions`, {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    Authorization: `Bearer ${process.env.TOKEN}`,
    "x-api-key": process.env.INAYA_API_KEY,
    "x-client-app-key": process.env.INAYA_APP_KEY,
  },
  body: JSON.stringify({
    required_modalities: ["Text"],
    selection_criteria: [
      { Model: { optionality: "MustHave", name: "Qwen3_VL_4B_Q4" } },
    ],
    system_prompt: "You are a helpful assistant. Answer in one short sentence.",
    temperature: 0.2,
  }),
});
console.log(await res.json());
```

**Expected response** — `200` with the session id and the selected model's
default image constraints (relevant when you later send images):

```json
{
  "ai_session_id": "01M38BVKKPGTB0D4YRRQD4BV7M",
  "image_size_constraints": { "max_supported": { "width": 1344, "height": 1344 }, "standard_size": { "width": 448, "height": 448 } }
}
```

> [!TIP]
> Criteria compose. A realistic production pick looks like
> `[{ "MinContextWindowLength": { "optionality": "MustHave", "length": 32768 } },
> { "ParameterCount": { "optionality": "NiceToHave", "count": "7B" } }]`
> — hard requirements plus preferences, letting the service choose the best
> available model. Pinning `Model` directly, as above, is best for tests and
> docs.

> [!PITFALL]
> `404` on session creation almost always means *no suitable model*: either the
> pinned model isn't pulled in the backend, or no model satisfies your
> `MustHave` criteria. Check the backend's model list before retrying.

## Step 2 — Send a message and get the full reply

Messages are `multipart/form-data` — the same call accepts a binary
`image_path` field later, so text rides along as a form field:

```bash verify=auto
curl -s -X POST "${BASE_URL}/v3/sessions/${SESSION_ID}/chat-messages" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -F "user_message=Say hello in exactly three words."
```

```python verify=auto
import json, os, urllib.request

boundary = "----g10boundary"
body = (
    f"--{boundary}\r\n"
    'Content-Disposition: form-data; name="user_message"\r\n\r\n'
    "Say hello in exactly three words.\r\n"
    f"--{boundary}--\r\n"
).encode()

req = urllib.request.Request(
    f"{os.environ['BASE_URL']}/v3/sessions/{os.environ['SESSION_ID']}/chat-messages",
    data=body,
    headers={
        "Content-Type": f"multipart/form-data; boundary={boundary}",
        "Authorization": f"Bearer {os.environ['TOKEN']}",
        "x-api-key": os.environ["INAYA_API_KEY"],
        "x-client-app-key": os.environ["INAYA_APP_KEY"],
    },
)
# Generous timeout: the first call of a session includes model-load time
print(json.dumps(json.load(urllib.request.urlopen(req, timeout=300)), indent=2))
```

```javascript verify=auto
const form = new FormData();
form.set("user_message", "Say hello in exactly three words.");

const res = await fetch(
  `${process.env.BASE_URL}/v3/sessions/${process.env.SESSION_ID}/chat-messages`,
  {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.TOKEN}`,
      "x-api-key": process.env.INAYA_API_KEY,
      "x-client-app-key": process.env.INAYA_APP_KEY,
    },
    body: form,
  },
);
console.log(await res.json());
```

**Expected response** — `200` with the complete assistant message (shape of
`response_time_ms` grows to the full generation time; the first call of a
session also includes model load):

```json
{
  "author": "Assistant",
  "content": [{ "Text": "Hello, how are you?" }],
  "creation_time": "2026-09-24T00:08:36Z",
  "id": "60OKQCPO89BJGC1IAGQL6C9JB90LGDQ1AT534L2E8K",
  "model_name": "Qwen3_VL_4B_Q4",
  "response_time_ms": 20695
}
```

> [!PITFALL]
> This endpoint has a generous internal timeout, but the route-level timeout
  does not cover streaming (see Step 3). If a non-streamed call returns `500`
  with an engine error after a long wait, the model likely failed to load —
  check the service logs and the backend's health before retrying.

## Step 3 — Stream the reply (SSE)

For interactive UIs, stream tokens as they generate:

```bash verify=auto
curl -s -N --max-time 180 -X POST \
  "${BASE_URL}/v3/sessions/${SESSION_ID}/streamed-chat-messages" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -F "user_message=Count from one to three."
```

**Expected response** — SSE events; concatenate the `Text` of every `message`
event until `done`:

```
event: message
data: {"author":"Assistant","content":[{"Text":"One"}],"creation_time":"…","id":"…","model_name":"Qwen3_VL_4B_Q4","response_time_ms":0}

event: message
data: {"author":"Assistant","content":[{"Text":", two"}],"creation_time":"…","id":"…","model_name":"Qwen3_VL_4B_Q4","response_time_ms":0}

event: done
data: true
```

> [!NOTE]
> Chunks are *deltas*, not cumulative — each `message` event carries only the
  newly generated fragment. Some fragments may be empty (e.g. reasoning markers
  with thinking models); skip them. An `event: error` terminates the stream on
  failure. The `--max-time` on the snippet is a safety cap for scripts, not a
  requirement.

## Step 4 — Review the conversation

History is stored server-side per session:

```bash verify=auto
curl -s "${BASE_URL}/v3/sessions/${SESSION_ID}/chat-history" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}"
```

```python verify=auto
import json, os, urllib.request

req = urllib.request.Request(
    f"{os.environ['BASE_URL']}/v3/sessions/{os.environ['SESSION_ID']}/chat-history",
    headers={
        "Authorization": f"Bearer {os.environ['TOKEN']}",
        "x-api-key": os.environ["INAYA_API_KEY"],
        "x-client-app-key": os.environ["INAYA_APP_KEY"],
    },
)
print(json.dumps(json.load(urllib.request.urlopen(req, timeout=60)), indent=2))
```

```javascript verify=auto
const res = await fetch(
  `${process.env.BASE_URL}/v3/sessions/${process.env.SESSION_ID}/chat-history`,
  {
    headers: {
      Authorization: `Bearer ${process.env.TOKEN}`,
      "x-api-key": process.env.INAYA_API_KEY,
      "x-client-app-key": process.env.INAYA_APP_KEY,
    },
  },
);
console.log(await res.json());
```

**Expected response** — `200` with every recorded message in order (user and
assistant):

```json
{ "messages": [ { "author": "User", "content": [{ "Text": "Say hello in exactly three words." }], "creation_time": "…", "id": "…" }, { "author": "Assistant", "content": [{ "Text": "Hello, how are you?" }], "creation_time": "…", "id": "…", "model_name": "Qwen3_VL_4B_Q4", "response_time_ms": 20695 } ] }
```

## Step 5 — End the session

```bash verify=auto
curl -s -o /dev/null -w "HTTP %{http_code}\n" -X DELETE \
  "${BASE_URL}/v3/sessions/${SESSION_ID}" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}"
```

**Expected response** — `200`; the session id and its history are discarded.

## Full working example

A complete create → chat → history → end cycle (`first_chat.sh`) — it fetches a
user JWT via the QA shortcut first (local/QA environments only; production uses
the G-01 flows):

```bash verify=auto
#!/usr/bin/env bash
set -euo pipefail

# 0. Obtain a user JWT (QA shortcut — see G-01 for production flows)
TOKEN=$(curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"device_id\": \"evg-demo-device\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"evg-demo\"}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])')
AUTH=(-H "Authorization: Bearer ${TOKEN}"
      -H "x-api-key: ${INAYA_API_KEY}"
      -H "x-client-app-key: ${INAYA_APP_KEY}")

# 1. Create a session
SESSION_ID=$(curl -sf -X POST "${BASE_URL}/v3/sessions" "${AUTH[@]}" \
  -H "Content-Type: application/json" \
  -d '{
    "required_modalities": ["Text"],
    "selection_criteria": [ { "Model": { "optionality": "MustHave", "name": "Qwen3_VL_4B_Q4" } } ],
    "temperature": 0.2
  }' | python3 -c 'import json,sys; print(json.load(sys.stdin)["ai_session_id"])')
echo "session: ${SESSION_ID}"

# 2. Chat (first call loads the model — allow time)
curl -sf -X POST "${BASE_URL}/v3/sessions/${SESSION_ID}/chat-messages" \
  "${AUTH[@]}" -F "user_message=Say hello in exactly three words."
echo

# 3. History
curl -sf "${BASE_URL}/v3/sessions/${SESSION_ID}/chat-history" "${AUTH[@]}" \
  | python3 -c 'import json,sys; d=json.load(sys.stdin); print("messages:", len(d["messages"]))'

# 4. End the session
curl -sf -X DELETE "${BASE_URL}/v3/sessions/${SESSION_ID}" "${AUTH[@]}" \
  && echo "session ended"
```

(`AUTH_URL` is the Auth Service base URL — `http://127.0.0.1:40004` on the local
stack; `TENANT_ID` is your tenant's id.)

## Try it live

Open the [Playground](../../playgrounds/playground.html) → **Applied
Intelligence** → *Evergreen AI · Session Console* — the same create/chat flow
with model-criteria controls, running against a live service.

## Copy agent prompt

```prompt
You are working with the Inaya Labs Evergreen AI Service at ${BASE_URL}. The
tenant key pair is in INAYA_API_KEY / INAYA_APP_KEY env vars; obtain a user JWT
via the QA login from guide G-01 if TOKEN is not already set. Steps: (1) POST
/v3/sessions with required_modalities ["Text"] and selection_criteria pinning
model "Qwen3_VL_4B_Q4" as MustHave; report the ai_session_id. (2) POST
/v3/sessions/<id>/chat-messages as multipart with field user_message "Say hello
in exactly three words."; report the reply text and model_name. Note the first
call may take tens of seconds while the model loads. (3) GET
/v3/sessions/<id>/chat-history and report the message count. (4) DELETE
/v3/sessions/<id> to end the session. Never log the JWT.
```

## API reference

- [Evergreen AI Service — Swagger UI](https://evergreen-sandbox.inayalabs.com/v3/swagger-ui/) · [Scalar](https://evergreen-sandbox.inayalabs.com/v3/scalar)

## Next journeys

- **Grounding answers in your documents** (G-11) — add `use_rag` to the session and ground replies in your own collections.
- **Choosing Evergreen vs Bella vs Chloe** (G-13) — which of the platform's three AI surfaces fits your use case.

### G-11 · Grounding answers in your documents
URL: /docs/guides/grounding-documents.html

> Upload documents to a RAG collection, query them with hybrid semantic search, and wire the collection into an Evergreen AI chat session.

> **Journey.** As a developer, I want my users' questions answered from *our own*
> documents — policies, runbooks, FAQs — so the AI cites our material instead of
> guessing.

The RAG Service is the platform's retrieval layer. It works in two modes:
**update mode** (add and remove documents in a collection) and **query mode**
(ask a natural-language question, get back the most relevant chunks with
citations). This guide walks the full loop — upload, list, search, delete — and
then wires the collection into an Evergreen AI chat session so replies are
grounded in your material. Verified against the local stack (RAG Service
`:40100`).

## Prerequisites

- The RAG Service is reachable (local stack `:40100`, or the shared dev
  subdomain).
- Your tenant key pair in `INAYA_API_KEY` / `INAYA_APP_KEY`, and a user JWT in
  `TOKEN` (obtain one via guide G-01; the full example below fetches one via the
  QA sign-in shortcut). On the local stack the service has no auth middleware of
  its own — the header trio is enforced by the platform gateway in deployed
  environments, and sending it everywhere keeps your code environment-agnostic.
- `curl`, or any HTTP client. Snippets also show Python and Node.
- A document to play with. `.txt` is the simplest; `.pdf`, `.docx`, `.pptx`,
  and `.xlsx` are also accepted.

## Concepts (only what this journey needs)

- **A collection is a named bucket of documents.** Every operation is scoped by
  `collection_id` — one tenant typically uses one collection per product,
  project, or knowledge base. Collections come into existence when you first
  upload into them; there is no separate "create collection" call.
- **Update mode = three calls.** `POST /v3/documents` (multipart upload) or
  `POST /v3/ingest` (raw bytes) adds a document; `GET /v3/documents` lists a
  collection; `DELETE /v3/documents/{id}` removes one.
- **Query mode = one call.** `POST /v3/search` takes a `question` and returns
  `chunks` (the best-matching passages, with scores and page/section metadata),
  `citations` (file + page + section to display alongside an answer), and
  `glossary_terms` (domain-term definitions to inject into a prompt).
- **Document ids are derived, not chosen.** Uploading `runbook.txt` into
  collection `onboarding_docs` produces the id `onboarding_docs-runbook.txt`.

> [!NOTE]
> The store is in-memory: ingested documents do not survive a service restart.
> Treat ingestion as part of your startup/bootstrap flow, and check the roadmap
> before relying on it as a system of record.

> [!NOTE]
> Two response fields are contractually stable but not yet populated in this
> build: `glossary_terms` (glossary extraction is a stub, so it is always
> `[]` today) and vector-similarity ranking (the embedding engine is a
> placeholder, so ranking is effectively keyword/BM25). The `include_glossary`
> flag and `max_rag_context_tokens` budget are accepted and forward-compatible.
> Keyword-style questions already rank well.

## Step 1 — Upload a document (update mode)

Multipart upload takes the file plus the target collection:

```bash verify=auto
curl -s -X POST "${BASE_URL}/v3/documents" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -F "file=@runbook.txt" \
  -F "collection_id=${COLLECTION}"
```

```python verify=auto
import json, os, urllib.request

boundary = "----g11boundary"
body = (
    f"--{boundary}\r\n"
    'Content-Disposition: form-data; name="file"; filename="runbook.txt"\r\n'
    "Content-Type: text/plain\r\n\r\n"
    "Acme Platform Onboarding Runbook\n\n"
    "Section 1: Tenant provisioning\n"
    "Rotate the apikey every ninety days; rotation never interrupts traffic.\r\n"
    f"--{boundary}\r\n"
    'Content-Disposition: form-data; name="collection_id"\r\n\r\n'
    f"{os.environ['COLLECTION']}\r\n"
    f"--{boundary}--\r\n"
).encode()

req = urllib.request.Request(
    f"{os.environ['BASE_URL']}/v3/documents",
    data=body,
    headers={
        "Content-Type": f"multipart/form-data; boundary={boundary}",
        "Authorization": f"Bearer {os.environ['TOKEN']}",
        "x-api-key": os.environ["INAYA_API_KEY"],
        "x-client-app-key": os.environ["INAYA_APP_KEY"],
    },
)
print(json.dumps(json.load(urllib.request.urlopen(req, timeout=60)), indent=2))
```

```javascript verify=auto
import { readFileSync } from "node:fs";

const form = new FormData();
form.set("file", new Blob([readFileSync("runbook.txt")]), "runbook.txt");
form.set("collection_id", process.env.COLLECTION);

const res = await fetch(`${process.env.BASE_URL}/v3/documents`, {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.TOKEN}`,
    "x-api-key": process.env.INAYA_API_KEY,
    "x-client-app-key": process.env.INAYA_APP_KEY,
  },
  body: form,
});
console.log(await res.json());
```

**Expected response** — `200` with the derived document id and the chunk count
produced by ingestion:

```json
{ "id": "onboarding_docs-runbook.txt", "collection_id": "onboarding_docs", "file_name": "runbook.txt", "chunk_count": 1 }
```

> [!TIP]
> Automations and pipelines without file handling can use the raw-bytes variant
> instead: `POST /v3/ingest?collection_id=…` with header `X-File-Name: runbook.txt`
> and the file as the `application/octet-stream` body. It returns
> `{ "document_id": "onboarding_docs-runbook.txt" }` — the same id under a
> different field name.

> [!PITFALL]
> **Do not put hyphens in collection ids (or file names).** The id is built as
> `{collection_id}-{file_name}` and the service parses the collection back out
> of the id on delete — with a hyphenated collection id, `DELETE` returns `200`
> but silently removes nothing. Use `snake_case`/`CamelCase` ids, and confirm
> removals with a list call (Step 3).

## Step 2 — Search the collection (query mode)

Ask a question; get back ranked chunks ready to paste into an LLM prompt (or
to feed straight to an Evergreen AI session — see the capstone below):

```bash verify=auto
curl -s -X POST "${BASE_URL}/v3/search" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "collection_id": "'"${COLLECTION}"'",
    "question": "how often should the apikey be rotated?",
    "include_glossary": true,
    "max_rag_context_tokens": 2000
  }'
```

```python verify=auto
import json, os, urllib.request

req = urllib.request.Request(
    f"{os.environ['BASE_URL']}/v3/search",
    data=json.dumps({
        "collection_id": os.environ["COLLECTION"],
        "question": "how often should the apikey be rotated?",
        "include_glossary": True,
        "max_rag_context_tokens": 2000,
    }).encode(),
    headers={
        "Content-Type": "application/json",
        "Authorization": f"Bearer {os.environ['TOKEN']}",
        "x-api-key": os.environ["INAYA_API_KEY"],
        "x-client-app-key": os.environ["INAYA_APP_KEY"],
    },
)
print(json.dumps(json.load(urllib.request.urlopen(req, timeout=60)), indent=2))
```

**Expected response** — `200`; chunks ranked by score, plus citations and the
(currently empty) glossary list:

```json
{
  "chunks": [
    { "chunk_id": "onboarding_docs:onboarding_docs-runbook.txt:0", "document_id": "onboarding_docs-runbook.txt", "file_name": "runbook.txt", "page_number": 1, "section_hierarchy": "Root", "content": "[Context Section: Root]\nAcme Platform Onboarding Runbook\n…", "score": 1.33 }
  ],
  "citations": [ { "file_name": "runbook.txt", "page_number": 1, "section": "Root" } ],
  "glossary_terms": []
}
```

> [!NOTE]
> `question` and `collection_id` are both required — either one empty gets a
> `400`. An unknown collection is not an error: you get `200` with empty
> `chunks`, which is how "nothing ingested yet" surfaces. (Also note the
> `chunk_count` field in *list* responses is always `0` in this build — rely on
> the upload response and actual search hits instead.)

## Step 3 — List, then remove a document (update mode)

List everything in the collection:

```bash verify=auto
curl -s "${BASE_URL}/v3/documents?collection_id=${COLLECTION}" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}"
```

**Expected response** — `200` with the collection's contents:

```json
{ "documents": [ { "id": "onboarding_docs-runbook.txt", "collection_id": "onboarding_docs", "file_name": "runbook.txt", "chunk_count": 0 } ], "total": 1 }
```

Delete by the derived id:

```bash verify=auto
curl -s -o /dev/null -w "HTTP %{http_code}\n" -X DELETE \
  "${BASE_URL}/v3/documents/${COLLECTION}-runbook.txt" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}"
```

**Expected response** — `200`, and a follow-up list no longer includes the
document.

> [!PITFALL]
> `DELETE` currently returns `200` even when the document id does not exist —
> it is not an existence check. If you need to know a removal actually
> happened, list before and after.

## Full working example

A complete update → query → cleanup cycle (`rag_tour.sh`) — it fetches a user
JWT via the QA shortcut first (local/QA environments only; production uses the
G-01 flows):

```bash verify=auto
#!/usr/bin/env bash
set -euo pipefail

# 0. Obtain a user JWT (QA shortcut — see G-01 for production flows)
TOKEN=$(curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"device_id\": \"rag-demo-device\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"rag-demo\"}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])')
AUTH=(-H "Authorization: Bearer ${TOKEN}"
      -H "x-api-key: ${INAYA_API_KEY}"
      -H "x-client-app-key: ${INAYA_APP_KEY}")

# 1. Upload (raw-bytes variant)
DOC_ID=$(curl -sf -X POST "${BASE_URL}/v3/ingest?collection_id=${COLLECTION}" \
  "${AUTH[@]}" -H "X-File-Name: runbook.txt" -H "Content-Type: application/octet-stream" \
  --data-binary @runbook.txt \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["document_id"])')
echo "ingested: ${DOC_ID}"

# 2. Query
curl -sf -X POST "${BASE_URL}/v3/search" "${AUTH[@]}" \
  -H "Content-Type: application/json" \
  -d "{\"collection_id\": \"${COLLECTION}\", \"question\": \"how often should the apikey be rotated?\"}" \
  | python3 -c 'import json,sys; d=json.load(sys.stdin); c=d["chunks"][0]; print("top hit:", c["file_name"], "score", round(c["score"],3), "| citations:", len(d["citations"]))'

# 3. List and clean up
curl -sf "${BASE_URL}/v3/documents?collection_id=${COLLECTION}" "${AUTH[@]}" \
  | python3 -c 'import json,sys; print("docs:", json.load(sys.stdin)["total"])'
curl -sf -o /dev/null -w "delete: %{http_code}\n" -X DELETE "${BASE_URL}/v3/documents/${DOC_ID}" "${AUTH[@]}"
```

(`AUTH_URL` is the Auth Service base URL — `http://127.0.0.1:40004` on the
local stack; `BASE_URL` here is the RAG Service — `http://127.0.0.1:40100`.)

## Capstone — ground an Evergreen AI chat session

Retrieval becomes "grounding" when a chat session uses it automatically.
Evergreen AI accepts a `use_rag` block at session creation; every message in
that session is then answered against your collection (the service needs
`RAG_SERVICE_URL` set — it is, on the local stack and in deployed
environments):

```bash verify=auto
curl -s -X POST "${EVG_URL}/v3/sessions" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "required_modalities": ["Text"],
    "selection_criteria": [ { "Model": { "optionality": "MustHave", "name": "Qwen3_VL_4B_Q4" } } ],
    "temperature": 0.2,
    "use_rag": { "collection_id": "'"${COLLECTION}"'", "include_glossary": true }
  }'
```

Chat messages on that session behave exactly as in guide G-10 — but the answer
now comes from your documents. With a short refund-policy text ingested into
the collection, asking *"According to the refund policy, how long do I have for
a full refund? One sentence."* produced:

```json
{ "author": "Assistant", "content": [{ "Text": "Full refunds are available within thirty days of purchase, no questions asked." }], "model_name": "Qwen3_VL_4B_Q4" }
```

(First call in a session still pays the model-load cost — see G-10.)

## Try it live

Open the [Playground](../../playgrounds/playground.html#ai) → **Applied
Intelligence** → *RAG* — upload a file, search the collection, and delete it,
running against a live service.

## Copy agent prompt

```prompt
You are working with the Inaya Labs RAG Service at ${BASE_URL} (local port
40100). The tenant key pair is in INAYA_API_KEY / INAYA_APP_KEY env vars;
obtain a user JWT via the QA login from guide G-01 if TOKEN is not already set.
Steps: (1) POST /v3/ingest?collection_id=onboarding_docs with header
X-File-Name: runbook.txt and the raw text bytes of the provided document;
report the document_id. (2) POST /v3/search with collection_id
onboarding_docs and question "how often should the apikey be rotated?";
report the top chunk's file_name and score. (3) GET /v3/documents?collection_id=
onboarding_docs and report the total. (4) DELETE /v3/documents/<document_id>
and confirm the collection is empty. Never use hyphens in collection ids.
Never log the JWT.
```

The same four operations are also exposed as authenticated MCP tools
(`ingest_document`, `list_documents`, `delete_document`, `search_documents`)
for agentic access — see guide G-02 for the discovery and calling mechanics.

## API reference

- [RAG Service — Swagger UI](https://rag-service-sandbox.inayalabs.com/v3/rag/swagger-ui/) · [Scalar](https://rag-service-sandbox.inayalabs.com/v3/rag/scalar/)

## Next journeys

- **Detecting objects and poses in images** (G-12) — the platform's vision API: upload an image, get back labeled boxes and keypoints.
- **Choosing Evergreen vs Bella vs Chloe** (G-13) — which of the platform's three AI surfaces fits your use case.
- **Your first AI chat session** (G-10) — the session mechanics this guide's capstone builds on.

### G-12 · Detecting objects and poses in images
URL: /docs/guides/detecting-objects-poses.html

> Upload an image and get back labeled bounding boxes for every object and 17-keypoint skeletons for every person, with relative coordinates and confidence scores.

> **Journey.** As a developer, I want to submit an image and learn what's in it —
> which objects, where, and how sure the model is — and for people, where their
> body keypoints are. No model hosting, no GPU wrangling.

The Object Detection Service runs YOLO-family vision models behind two calls:
`POST /v3/detect` labels every object in an image with a bounding box;
`POST /v3/pose` finds every person and returns their 17 body keypoints. This
guide walks both, explains the coordinate system, and shows client-side
confidence filtering. Verified against the local stack (Object Detection
Service `:45091`, YOLO26 small).

## Prerequisites

- The Object Detection Service is reachable (local stack `:45091`, or the
  shared dev subdomain).
- Your tenant key pair in `INAYA_API_KEY` / `INAYA_APP_KEY`, and a user JWT in
  `TOKEN` (obtain one via guide G-01; the full example below fetches one via the
  QA sign-in shortcut). On the local stack the REST routes have no auth
  middleware of their own — the header trio is enforced by the platform gateway
  in deployed environments, and sending it everywhere keeps your code
  environment-agnostic.
- `curl`, or any HTTP client. Snippets also show Python and Node.
- An image file (PNG/JPEG, at most 25 MB). The examples below assume it is in
  `${IMAGE_PATH}`.

## Concepts (only what this journey needs)

- **One image in, JSON out.** Both endpoints accept `multipart/form-data` with
  the file in a field named `image`, and return JSON. `detect` covers the COCO
  object set (person, car, dog, chair, …); `pose` returns people only.
- **All coordinates are relative (0.0–1.0).** Bounding boxes are
  `x1, y1` (top-left) → `x2, y2` (bottom-right), each relative to the image
  width/height. Keypoint `x`/`y` work the same way. To draw on the image,
  multiply by its pixel dimensions — no image-size math needed to *compare*
  detections across images.
- **Confidence filtering is your job on REST.** The endpoints return every
  detection; you threshold `confidence` yourself (the MCP tools accept a
  `min_confidence` argument instead).
- **The models initialize lazily.** The first request after service start pays
  a one-time model-load cost (seconds), then each image costs roughly a second
  on the local stack.
- **Every inference is snapshotted** to the service's Postgres store; the MCP
  tool `get_recent_detections` reads the latest ten. Useful for audit trails —
  be deliberate about what images you send.

## Step 1 — Detect objects

```bash verify=auto
curl -s -X POST "${BASE_URL}/v3/detect" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -F "image=@${IMAGE_PATH}"
```

```python verify=auto
import json, os, urllib.request

boundary = "----g12boundary"
image = open(os.environ["IMAGE_PATH"], "rb").read()
body = (
    f"--{boundary}\r\n"
    'Content-Disposition: form-data; name="image"; filename="image.png"\r\n'
    "Content-Type: image/png\r\n\r\n"
).encode() + image + f"\r\n--{boundary}--\r\n".encode()

req = urllib.request.Request(
    f"{os.environ['BASE_URL']}/v3/detect",
    data=body,
    headers={
        "Content-Type": f"multipart/form-data; boundary={boundary}",
        "Authorization": f"Bearer {os.environ['TOKEN']}",
        "x-api-key": os.environ["INAYA_API_KEY"],
        "x-client-app-key": os.environ["INAYA_APP_KEY"],
    },
)
print(json.dumps(json.load(urllib.request.urlopen(req, timeout=120)), indent=2))
```

```javascript verify=auto
import { readFileSync } from "node:fs";

const form = new FormData();
form.set("image", new Blob([readFileSync(process.env.IMAGE_PATH)]), "image.png");

const res = await fetch(`${process.env.BASE_URL}/v3/detect`, {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.TOKEN}`,
    "x-api-key": process.env.INAYA_API_KEY,
    "x-client-app-key": process.env.INAYA_APP_KEY,
  },
  body: form,
});
console.log(await res.json());
```

**Expected response** — `200` with every detected object (verified against a
1280×720 street photo; trimmed):

```json
{
  "detections": [
    { "bounding_box": { "x1": 0.155, "y1": 0.118, "x2": 0.325, "y2": 0.941 }, "confidence": 0.961, "object_instance_id": 0, "object_type": "person" },
    { "bounding_box": { "x1": 0.522, "y1": 0.359, "x2": 0.999, "y2": 0.912 }, "confidence": 0.968, "object_instance_id": 0, "object_type": "car" }
  ],
  "processing_time_ms": 1083
}
```

> [!TIP]
> Pixel conversion is one multiply away — on the verified example, the person
> box lands at x 198–416, y 85–678 on the 1280×720 image:
> `px_x1 = det["bounding_box"]["x1"] * img_width`, likewise for the rest.
> `object_instance_id` is a per-response tracker id: REST calls are stateless,
> so don't expect ids to match between two uploads of the same scene.

> [!PITFALL]
> Error bodies are `{ "error": "…" }` with distinct codes: `400` — the
> multipart form has no `image` field; `422` — the bytes aren't a decodable
> image (`"The image format could not be determined"`); `503` — the model
> couldn't initialize (e.g. `OBJECT_DETECTION_GENERAL_MODEL_PATH` points at a
> missing file). All three were observed live; treat `503` as
> infrastructure, not input, and check the service logs.

## Step 2 — Estimate poses

Same multipart contract, keypoints instead of labels — up to 17 COCO joints
per person (`nose`, `left_eye`, …, `left_ankle`), each with its own
confidence:

```bash verify=auto
curl -s -X POST "${BASE_URL}/v3/pose" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -F "image=@${IMAGE_PATH}"
```

```python verify=auto
import json, os, urllib.request

boundary = "----g12boundary"
image = open(os.environ["IMAGE_PATH"], "rb").read()
body = (
    f"--{boundary}\r\n"
    'Content-Disposition: form-data; name="image"; filename="image.png"\r\n'
    "Content-Type: image/png\r\n\r\n"
).encode() + image + f"\r\n--{boundary}--\r\n".encode()

req = urllib.request.Request(
    f"{os.environ['BASE_URL']}/v3/pose",
    data=body,
    headers={
        "Content-Type": f"multipart/form-data; boundary={boundary}",
        "Authorization": f"Bearer {os.environ['TOKEN']}",
        "x-api-key": os.environ["INAYA_API_KEY"],
        "x-client-app-key": os.environ["INAYA_APP_KEY"],
    },
)
d = json.load(urllib.request.urlopen(req, timeout=120))
for pose in d["poses"]:
    visible = [k["part"] for k in pose["keypoints"] if k["confidence"] >= 0.5]
    print(f"person conf={pose['confidence']:.3f} visible keypoints: {', '.join(visible)}")
print("processing_time_ms:", d["processing_time_ms"])
```

**Expected response** — `200` with one pose per person (verified on the same
street photo — one person, 17 keypoints returned, 16 at confidence ≥ 0.5;
trimmed):

```json
{
  "poses": [
    {
      "bounding_box": { "x1": 0.155, "y1": 0.120, "x2": 0.324, "y2": 0.939 },
      "confidence": 0.926,
      "object_instance_id": 0,
      "keypoints": [
        { "index": 0, "part": "nose", "x": 0.234, "y": 0.157, "confidence": 0.995 },
        { "index": 5, "part": "left_shoulder", "x": 0.177, "y": 0.276, "confidence": 0.98 }
      ]
    }
  ],
  "processing_time_ms": 1103
}
```

> [!NOTE]
> All 17 keypoints are always present — there is no server-side visibility
> cut. A keypoint with very low confidence (below ~0.1) is probably not
> visible in the image; threshold before you draw a skeleton or compute
> angles.

## Step 3 — Filter by confidence (client-side)

The REST response is unfiltered, so production UIs apply their own threshold.
For example, keeping `person` and `car` detections at 0.5 or better:

```python verify=auto
import json, os, urllib.request

boundary = "----g12boundary"
image = open(os.environ["IMAGE_PATH"], "rb").read()
body = (
    f"--{boundary}\r\n"
    'Content-Disposition: form-data; name="image"; filename="image.png"\r\n'
    "Content-Type: image/png\r\n\r\n"
).encode() + image + f"\r\n--{boundary}--\r\n".encode()

req = urllib.request.Request(
    f"{os.environ['BASE_URL']}/v3/detect",
    data=body,
    headers={"Content-Type": f"multipart/form-data; boundary={boundary}"},
)
d = json.load(urllib.request.urlopen(req, timeout=120))
keep = [x for x in d["detections"] if x["confidence"] >= 0.5 and x["object_type"] in ("person", "car")]
print(f"kept {len(keep)} of {len(d['detections'])} detections at >= 0.5")
for x in keep:
    print(" ", x["object_type"], round(x["confidence"], 3))
```

**Expected output** (verified on the street photo):

```
kept 5 of 8 detections at >= 0.5
  car 0.968
  person 0.961
  car 0.884
  car 0.831
  car 0.814
```

> [!TIP]
> If an AI agent drives this flow, prefer the service's MCP tools
> (`detect_objects`, `estimate_pose`): they take `image_base64` plus a
> server-side `min_confidence` filter, and `get_recent_detections` replays the
  latest snapshots. See guide G-02 for MCP mechanics.

## Full working example

A complete detect → filter → pose cycle (`vision_tour.sh`) — it fetches a user
JWT via the QA shortcut first (local/QA environments only; production uses the
G-01 flows):

```bash verify=auto
#!/usr/bin/env bash
set -euo pipefail

# 0. Obtain a user JWT (QA shortcut — see G-01 for production flows)
TOKEN=$(curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"device_id\": \"vision-demo-device\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"vision-demo\"}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])')
AUTH=(-H "Authorization: Bearer ${TOKEN}"
      -H "x-api-key: ${INAYA_API_KEY}"
      -H "x-client-app-key: ${INAYA_APP_KEY}")

# 1. Detect
curl -sf -X POST "${BASE_URL}/v3/detect" "${AUTH[@]}" -F "image=@${IMAGE_PATH}" \
  | python3 -c 'import json,sys; d=json.load(sys.stdin); print("detections:", len(d["detections"]), "in", d["processing_time_ms"], "ms")'

# 2. Pose
curl -sf -X POST "${BASE_URL}/v3/pose" "${AUTH[@]}" -F "image=@${IMAGE_PATH}" \
  | python3 -c 'import json,sys; d=json.load(sys.stdin); print("poses:", len(d["poses"]), "| keypoints on first pose:", len(d["poses"][0]["keypoints"]) if d["poses"] else 0)'
```

(`AUTH_URL` is the Auth Service base URL — `http://127.0.0.1:40004` on the
local stack; `BASE_URL` here is the Object Detection Service —
`http://127.0.0.1:45091`.)

## Try it live

Open the [Playground](../../playgrounds/playground.html#ai) → **Applied
Intelligence** → *Object Detection* — upload an image and see the boxes and
keypoints rendered, running against a live service.

## Copy agent prompt

```prompt
You are working with the Inaya Labs Object Detection Service at ${BASE_URL}
(local port 45091). The tenant key pair is in INAYA_API_KEY / INAYA_APP_KEY env
vars; obtain a user JWT via the QA login from guide G-01 if TOKEN is not
already set. Steps: (1) POST /v3/detect with multipart field "image" set to
the image at IMAGE_PATH; report the count of detections and each
object_type with its confidence. (2) Re-list only detections with
confidence >= 0.5. (3) POST /v3/pose on the same image; report the number of
people and, for the first person, how many of the 17 keypoints have
confidence >= 0.5. Remember coordinates are relative (0.0-1.0). Never log
the JWT.
```

## API reference

- [Object Detection Service — Swagger UI](https://object-detection-service-sandbox.inayalabs.com/v3/object-detection/swagger-ui/) · [Scalar](https://object-detection-service-sandbox.inayalabs.com/v3/object-detection/scalar/)

## Next journeys

- **Choosing Evergreen vs Bella vs Chloe** (G-13) — which of the platform's three AI surfaces fits your use case.
- **Your first AI chat session** (G-10) — text generation with model-selection criteria.
- **Grounding answers in your documents** (G-11) — retrieval-augmented generation over your own collections.

### G-13 · Choosing Evergreen vs Bella vs Chloe
URL: /docs/guides/choosing-ai-surface.html

> Three AI surfaces, one platform — sessions you configure, sessions pre-tuned for you, and impromptu reasoning that trades budget for depth.

> **Journey.** As a developer, I need an AI feature — but which of the
> platform's three AI surfaces do I call? This guide maps the decision, then
> shows the one call that defines each surface.

The platform offers three ways to get model intelligence, all backed by the
same Evergreen AI model fleet:

| Surface | What it is | You bring | Best for |
|---|---|---|---|
| **Evergreen AI** (session console) | A conversation *you* configure: model-selection criteria, generation parameters, RAG grounding, streaming | Session criteria and message flow | Built-in AI features in your product — chat, document Q&A, multimodal pipelines |
| **Bella** (predefined sessions) | The same session machinery, pre-tuned: an optimized system prompt and model pick loaded from a template | A template id and messages | Product-ready assistants (support personas, reviewers, companions) without prompt engineering |
| **Chloe** (reasoning API) | An OpenAI-compatible endpoint that plans how much reasoning a prompt deserves and spends accordingly | One question per call | Hard questions where answer quality matters more than per-call latency — analysis, comparison, judgment calls |

The short version: **Evergreen when you want control, Bella when you want a
tuned assistant without the tuning, Chloe when one shot must be as good as it
can be within a budget.** All three share the platform's auth (guide G-01) and
model governance; none of them require you to host anything.

## Concepts (only what this journey needs)

- **Sessions vs one-shot.** Evergreen and Bella are *session-based*: you create
  a session, exchange messages over it (history is server-side), and delete it
  when done. Chloe is *one-shot*: every call is self-contained — send messages,
  get the answer, nothing to clean up.
- **Criteria vs templates.** Evergreen sessions are built from
  `selection_criteria` (capability filters the service resolves to a model —
  see G-10). A predefined session replaces that whole block with one template
  id; the template's author already made the choices.
- **Effort is Chloe's budget knob.** Chloe's `effort` field (`fast`, `balanced`,
  `deep`, `maximum`, default `auto`) trades latency/cost against answer depth.
  Higher effort means more deliberation before the answer comes back — choose
  per call, not per integration.

## Path A — Evergreen AI: the configurable session

The full lifecycle (create → chat → stream → history → delete) is guide G-10;
the one call that defines this surface is session creation with your criteria:

```bash verify=auto
curl -s -X POST "${EVG_URL}/v3/sessions" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "required_modalities": ["Text"],
    "selection_criteria": [
      { "MinContextWindowLength": { "optionality": "MustHave", "length": 32768 } }
    ],
    "temperature": 0.2
  }'
```

**Expected response** — `200` with an `ai_session_id` you then address with the
chat calls from G-10.

> [!TIP]
> Add `"use_rag": { "collection_id": "…" }` to the creation body and the
> session grounds its answers in your own documents — see G-11.

## Path B — Bella: the predefined session

Same session machinery, one template id instead of a criteria block. Templates
ship with the platform (e.g. `igentic.bella.v1` — the Bella assistant persona)
or with your tenant's deployment; the id is the JSON file name in the
predefined-sessions folder, minus `.json`:

```bash verify=auto
curl -s -X POST "${EVG_URL}/v3/sessions/pre-defined" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Content-Type: application/json" \
  -d '{ "predefined_session_id": "'"${PREDEFINED_SESSION_ID}"'" }'
```

```python verify=auto
import json, os, urllib.request

req = urllib.request.Request(
    f"{os.environ['EVG_URL']}/v3/sessions/pre-defined",
    data=json.dumps({
        "predefined_session_id": os.environ["PREDEFINED_SESSION_ID"],
    }).encode(),
    headers={
        "Content-Type": "application/json",
        "Authorization": f"Bearer {os.environ['TOKEN']}",
        "x-api-key": os.environ["INAYA_API_KEY"],
        "x-client-app-key": os.environ["INAYA_APP_KEY"],
    },
)
print(json.dumps(json.load(urllib.request.urlopen(req, timeout=60)), indent=2))
```

**Expected response** — `200` with the same `ai_session_id` shape as Path A;
from here, chat/stream/history/delete work exactly as in G-10. Overlay fields
(`temperature`, `history_ttl_in_seconds`, …) may be sent alongside the id to
adjust the template per session.

> [!NOTE]
> A template can only start if its pinned model is actually available. The
> shipped templates pin cloud models; the local stack ships a docs template
> (`inayalabs.docsdemo.v1`, pinning the local `Qwen3_VL_4B_Q4`) so this path
> is exercisable offline. `404` means *template or its model not found*.

## Path C — Chloe: the reasoning endpoint

One call, no session. The request is shaped like an OpenAI chat completion;
`effort` is the budget knob:

```bash verify=manual
curl -s -X POST "${CHLOE_URL}/v3/chloe/chat/completions" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "chloe",
    "messages": [ { "role": "user", "content": "Which is the better trade-off for a logging library: structured JSON or plain text? Two sentences." } ],
    "effort": "balanced"
  }'
```

**Expected response** — `200` with an OpenAI-compatible body: `choices[0].message.content`
is the answer, plus a `deliberation` block recording what the request cost
(latency, estimated unit cost, effort honored). Treat `deliberation` as
telemetry, not input. Roles `system` and `developer` are accepted for steering;
`stream` is not yet supported — omit it. Evergreen unreachable → `503`.

> [!NOTE]
> Verified `manual`: the request/response contract, auth (`401` without a JWT),
> and validation errors (`400` for `stream: true`, empty messages, unsupported
> roles) were all exercised live, but the happy-path answer requires a model
> fleet member tagged with a *maturity* level — no locally-managed model
> carries one yet, so Chloe's profiles find no candidate on the local stack
> (the call fails with a gateway `404`, surfaced as `503`). On deployed
> environments with mature cloud models in the fleet, the same call answers
> normally.

## Full working example

Both session paths in one script (`which_surface.sh`) — it fetches a user JWT
via the QA shortcut first (local/QA environments only; production uses the
G-01 flows):

```bash verify=auto
#!/usr/bin/env bash
set -euo pipefail

# 0. Obtain a user JWT (QA shortcut — see G-01 for production flows)
TOKEN=$(curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"device_id\": \"g13-demo-device\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"g13-demo\"}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])')
AUTH=(-H "Authorization: Bearer ${TOKEN}"
      -H "x-api-key: ${INAYA_API_KEY}"
      -H "x-client-app-key: ${INAYA_APP_KEY}")

# 1. Path B — predefined session (Bella path)
SESSION_ID=$(curl -sf -X POST "${EVG_URL}/v3/sessions/pre-defined" "${AUTH[@]}" \
  -H "Content-Type: application/json" \
  -d "{\"predefined_session_id\": \"${PREDEFINED_SESSION_ID}\"}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["ai_session_id"])')
echo "predefined session: ${SESSION_ID}"

# 2. Chat on it, then clean up (same calls as G-10)
curl -sf --max-time 300 -X POST "${EVG_URL}/v3/sessions/${SESSION_ID}/chat-messages" \
  "${AUTH[@]}" -F "user_message=Say hello in exactly three words." \
  | python3 -c 'import json,sys; d=json.load(sys.stdin); print("reply:", d["content"][0]["Text"][:40], "| model:", d["model_name"])'
curl -sf -o /dev/null -w "session delete: %{http_code}\n" -X DELETE "${EVG_URL}/v3/sessions/${SESSION_ID}" "${AUTH[@]}"
```

(`AUTH_URL` is the Auth Service — `http://127.0.0.1:40004`; `EVG_URL` is
Evergreen AI — `http://127.0.0.1:40071`. `PREDEFINED_SESSION_ID` is
`inayalabs.docsdemo.v1` on the local stack, a shipped template id like
`igentic.bella.v1` in deployed environments.)

## Try it live

Open the [Playground](../../playgrounds/playground.html#ai) → **Applied
Intelligence**: *Evergreen AI · Session Console* for Path A, or the
**Our AI Models** family for Bella and Chloe chat UIs — same surfaces,
running against live services.

## Copy agent prompt

```prompt
You are helping pick between the Inaya Labs AI surfaces. Evergreen AI
(${EVG_URL}) creates configurable sessions (POST /v3/sessions with
selection_criteria — see guide G-10). Predefined sessions (Bella path) start
with POST /v3/sessions/pre-defined and a predefined_session_id such as
igentic.bella.v1. Chloe (${CHLOE_URL}) answers one-shot questions via POST
/v3/chloe/chat/completions with an OpenAI-style messages array and an effort
budget knob (fast|balanced|deep|maximum|auto). Steps: (1) start a predefined
session using PREDEFINED_SESSION_ID from env; report the ai_session_id.
(2) Send "Say hello in exactly three words." via POST
/v3/sessions/<id>/chat-messages (multipart field user_message); report the
reply text and model_name. (3) DELETE /v3/sessions/<id>. Never log the JWT.
```

## API reference

- [Evergreen AI Service — Swagger UI](https://evergreen-sandbox.inayalabs.com/v3/swagger-ui/) · [Scalar](https://evergreen-sandbox.inayalabs.com/v3/scalar) — sessions (custom and predefined)
- [Bella — Swagger UI](https://bella-service-sandbox.inayalabs.com/v3/swagger-ui/) — the Bella assistant facade
- [Chloe — Swagger UI](https://chloe-service-sandbox.inayalabs.com/v3/chloe/swagger-ui/) — the reasoning endpoint

## Next journeys

- **Your first AI chat session** (G-10) — the full Evergreen session lifecycle this guide's Path A builds on.
- **Chatting with Bella** (G-20) — the predefined-session surface in depth.
- **Chatting with Chloe** (G-21) — the reasoning endpoint in depth.
- **Grounding answers in your documents** (G-11) — add `use_rag` to any Evergreen-family session.

### G-20 · Chatting with Bella
URL: /docs/guides/chatting-with-bella.html

> One OpenAI-compatible call to a warm, pre-tuned assistant — persona, model pick, and safety posture already handled. Bring only a message.

> **Journey.** As a developer, I want a friendly assistant in my product —
> already persona-tuned, already safety-configured — without writing a system
> prompt or picking a model. This guide shows the one call that defines the
> Bella surface, plus the variations (images, streaming, roles) you will
> actually use.

Bella is the platform's curated-persona chat surface. Everything that makes the
assistant *the assistant* — the persona prompt, the model selection, the
generation defaults — lives server-side in a predefined session template
(`igentic.bella.v1`). Your integration is a single OpenAI-compatible endpoint
that is stateless: **every call is a fresh, one-shot conversation.** There is
no session id to create, keep, or delete, and nothing to clean up after a call.

## Concepts (only what this journey needs)

- **One call per conversation.** The service begins the predefined session,
  sends your message, collects the reply, and ends the session — all inside
  your one HTTP call. That makes Bella trivially scalable, but it also means
  **Bella has no memory**: each request is answered on its own.
- **Persona is server-side.** You don't send a system prompt in production;
  the template carries it. A `system`/`developer` message you *do* send is
  treated as per-call steering layered on top of the persona.
- **OpenAI-compatible shape.** The request and response mirror the OpenAI
  chat-completions contract — `messages`, `choices[0].message.content`,
  `stream` — so existing OpenAI client code points at Bella with a URL swap.
- **The effort knob is Chloe's, not Bella's.** Budget/depth control lives on
  the reasoning endpoint (guide G-21). Bella tunes for a consistent,
  product-ready voice instead.

## The one call

A text question, answered:

```bash verify=auto
curl -s --max-time 280 -X POST "${BELLA_URL}/v3/bella/chat/completions" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "inaya-bella",
    "messages": [ { "role": "user", "content": "Give me one kind habit to start this week." } ]
  }'
```

**Expected response** — `200` with an OpenAI-shaped body:

```json
{
  "id": "chatcmpl-…",
  "object": "chat.completion",
  "created": 1790223878,
  "model": "GptOss_120B_Cloud",
  "choices": [
    { "index": 0,
      "message": { "role": "assistant", "content": "**Hello, dear friend**" },
      "finish_reason": "stop" }
  ],
  "usage": { "prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0 }
}
```

The answer is `choices[0].message.content`. Two fields to read correctly:
`model` names the *fleet model* that actually answered (the template's pinned
pick — useful for support tickets), and `usage` is a placeholder (all zeros)
reserved for a future metering release. `model` in the request is optional and
purely a client label; Bella always answers with the persona's model.

## Images: multipart content

Attach an image with an `image_url` content part. The `url` may be a
`data:image/…;base64,…` URI or an `http(s)` URL the service fetches for you:

```bash verify=auto
python3 - <<'PY'
import base64, io, json
from PIL import Image, ImageDraw

img = Image.new("RGB", (256, 256), (240, 240, 240))
draw = ImageDraw.Draw(img)
draw.ellipse((60, 60, 196, 196), fill=(200, 30, 30))
buf = io.BytesIO()
img.save(buf, format="PNG")
data_uri = "data:image/png;base64," + base64.b64encode(buf.getvalue()).decode()

body = {
    "model": "inaya-bella",
    "messages": [{
        "role": "user",
        "content": [
            { "type": "text", "text": "Describe this image in one short sentence." },
            { "type": "image_url", "image_url": { "url": data_uri } },
        ],
    }],
}
open("bella_image_request.json", "w").write(json.dumps(body))
print("request written")
PY

curl -s --max-time 280 -X POST "${BELLA_URL}/v3/bella/chat/completions" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Content-Type: application/json" \
  -d @bella_image_request.json
```

**Expected response** — `200`, same body shape as text-only. The image is
accepted and forwarded to the model; how well the reply uses it depends on the
model pinned by the persona's template — a text-only fleet model answers
without seeing it. The text part can also be a plain string when there is no
image, exactly as in the first example.

## Multi-turn appearance, one-shot reality

Clients that keep chat history (including the platform Playground) send the
whole transcript — user and assistant turns — on every call. Bella accepts
that shape: prior `assistant` messages are allowed and ignored, and a
`system`/`developer` message is applied as per-call steering. Only the current
`user` text is actually answered:

```bash verify=auto
curl -s --max-time 280 -X POST "${BELLA_URL}/v3/bella/chat/completions" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "inaya-bella",
    "messages": [
      { "role": "system", "content": "Answer in one word." },
      { "role": "user", "content": "Hi" },
      { "role": "assistant", "content": "Hello! How can I help?" },
      { "role": "user", "content": "Pick a color." }
    ]
  }'
```

**Expected response** — `200`. The service does not reject transcript-shaped
bodies, but it does not remember them either. If your product needs real
server-side memory, use an Evergreen session (guide G-10) or start a
predefined session yourself (guide G-13, Path B) instead of this facade.

## Streaming

Set `stream: true` and the endpoint returns an SSE stream of
`chat.completion.chunk` events — a role delta first, content deltas next, a
final `finish_reason: "stop"` chunk, then `data: [DONE]`:

```bash verify=auto
curl -s -N --max-time 280 -X POST "${BELLA_URL}/v3/bella/chat/completions" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "inaya-bella",
    "stream": true,
    "messages": [ { "role": "user", "content": "Count from one to three, separated by spaces." } ]
  }' | grep '^data:' | head -n 6
```

**Expected output** — SSE `data:` lines, e.g. a `{"delta":{"role":"assistant"}}`
chunk, one or more `{"delta":{"content":"…"}}` chunks, a
`{"delta":{},"finish_reason":"stop"}` chunk, and finally `data: [DONE]`. In
streaming mode the `model` field on chunks echoes your request label; the
non-streaming response's `model` field names the fleet model that answered.

## Error contract

Three failures cover almost everything that goes wrong:

```bash verify=auto
KEYS=(-H "x-api-key: ${INAYA_API_KEY}" -H "x-client-app-key: ${INAYA_APP_KEY}")

echo "== missing JWT =="
curl -s -o /tmp/bella_err1.json -w "%{http_code}\n" -X POST "${BELLA_URL}/v3/bella/chat/completions" \
  "${KEYS[@]}" -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"hi"}]}'
cat /tmp/bella_err1.json; echo

echo "== unsupported role =="
curl -s -w "\n%{http_code}\n" -X POST "${BELLA_URL}/v3/bella/chat/completions" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}" -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"tool","content":"hi"}]}'

echo "== empty messages =="
curl -s -w "\n%{http_code}\n" -X POST "${BELLA_URL}/v3/bella/chat/completions" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}" -H "Content-Type: application/json" \
  -d '{"messages":[]}'
```

**Expected** — `401` with `{"error":"Missing Authorization token"}` for a
missing/invalid JWT; `400` with `Invalid request: Unsupported role 'tool'.
Only 'user', 'assistant', 'system', and 'developer' are supported.` for a bad
role; `400` with `Invalid request: messages array must not be empty` for an
empty array. Evergreen AI unreachable at answer time → `500` with an
`AI chat failed: …` message — treat it as retryable.

> [!NOTE]
> Accepted-but-inert fields: `temperature` and `max_tokens` parse but are
> ignored (the persona template owns generation defaults), and an `effort`
> field is silently ignored — effort budgeting is Chloe's surface (G-21).

## Full working example

One script (`bella_chat.sh`) — QA-login, a text turn, an image turn, and a
streamed turn, each asserted (`AUTH_URL` is the Auth Service —
`http://127.0.0.1:40004`; `BELLA_URL` is Bella — `http://127.0.0.1:40005`;
`TENANT_ID` is your sandbox tenant):

```bash verify=auto
#!/usr/bin/env bash
set -euo pipefail

# 0. Obtain a user JWT (QA shortcut — see G-01 for production flows)
TOKEN=$(curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"device_id\": \"g20-demo-device\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"g20-demo\"}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])')
AUTH=(-H "Authorization: Bearer ${TOKEN}"
      -H "x-api-key: ${INAYA_API_KEY}"
      -H "x-client-app-key: ${INAYA_APP_KEY}")

# 1. Text turn — assert the OpenAI response shape
curl -sf --max-time 280 -X POST "${BELLA_URL}/v3/bella/chat/completions" "${AUTH[@]}" \
  -H "Content-Type: application/json" \
  -d '{"model":"inaya-bella","messages":[{"role":"user","content":"Say hello in exactly three words."}]}' \
  | python3 -c '
import json, sys
d = json.load(sys.stdin)
assert d["object"] == "chat.completion", d
assert d["choices"][0]["finish_reason"] == "stop", d
assert d["choices"][0]["message"]["role"] == "assistant"
assert d["choices"][0]["message"]["content"].strip(), "empty reply"
print("text turn: ok | model:", d["model"])'

# 2. Transcript-shaped body (assistant history + system steering) is accepted
curl -sf --max-time 280 -o /dev/null -w "transcript turn: %{http_code}\n" \
  -X POST "${BELLA_URL}/v3/bella/chat/completions" "${AUTH[@]}" \
  -H "Content-Type: application/json" \
  -d '{"model":"inaya-bella","messages":[{"role":"user","content":"Hi"},{"role":"assistant","content":"Hello!"},{"role":"user","content":"Bye"}]}'

# 3. Streamed turn — collect deltas until [DONE]
CHUNKS=$(curl -s -N --max-time 280 -X POST "${BELLA_URL}/v3/bella/chat/completions" "${AUTH[@]}" \
  -H "Content-Type: application/json" \
  -d '{"model":"inaya-bella","stream":true,"messages":[{"role":"user","content":"Count from one to three, separated by spaces."}]}')
echo "${CHUNKS}" | grep -q 'data: \[DONE\]' && echo "stream turn: ok (terminated with [DONE])"
```

## Try it live

Open the [Playground](../../playgrounds/playground.html#ai-models) →
**Our AI Models**: *Bella* is a fully functional chat UI on this exact
endpoint — suggestions, image attachments, and streaming included.

## Copy agent prompt

```prompt
You are helping a developer call the Bella chat endpoint (${BELLA_URL}).
Bella is a stateless, OpenAI-compatible chat surface: POST
/v3/bella/chat/completions with header Authorization: Bearer ${TOKEN} plus the
x-api-key / x-client-app-key pair. Body: optional model label ("inaya-bella"),
messages array (roles user, assistant, system, developer; content either a
plain string or parts of type text / image_url where image_url.url is a
data:image/...;base64 URI or an http(s) URL), optional stream:true for SSE
chunks ending in "data: [DONE]"). Each call is a fresh one-shot session — the
assistant role in history is accepted but ignored; Bella has no memory.
Steps: (1) POST one user message and report choices[0].message.content and the
model field. (2) Repeat with stream:true and count the content deltas before
[DONE]. Errors: 401 {"error":"Missing Authorization token"}; 400 for
unsupported roles or empty messages. Never log the JWT.
```

## API reference

- [Bella — Swagger UI](https://bella-service-sandbox.inayalabs.com/v3/swagger-ui/) · [Scalar](https://bella-service-sandbox.inayalabs.com/v3/bella/scalar) — the curated-persona chat facade

## Next journeys

- **Choosing Evergreen vs Bella vs Chloe** (G-13) — where this surface fits among the platform's three AI surfaces.
- **Chatting with Chloe** (G-21) — the reasoning endpoint: one-shot calls with a budget knob instead of a persona.
- **Your first AI chat session** (G-10) — full server-side conversation memory via Evergreen sessions, when Bella's one-shot model is not enough.

### G-21 · Chatting with Chloe
URL: /docs/guides/chatting-with-chloe.html

> One OpenAI-compatible call where the service decides how much reasoning your question deserves — a budget knob trades cost and latency against answer depth.

> **Journey.** As a developer, I have a hard question — an analysis, a
> comparison, a judgment call — and I want the best answer the platform can
> produce within a budget I control. This guide shows the one call that defines
> the Chloe surface and the budget knob that shapes every answer.

Chloe is the platform's reasoning surface. You send an OpenAI-compatible chat
request; the service decides how much deliberation the prompt deserves, spends
it, and returns the answer with a telemetry block recording what the request
cost. Like Bella (guide G-20), Chloe is **stateless one-shot**: every call is
self-contained — the service creates a fresh, single-use model session for the
call and discards it afterwards. There is no session id, no history, and
nothing to clean up.

## Concepts (only what this journey needs)

- **The effort knob is the product.** `effort` (`fast`, `balanced`, `deep`,
  `maximum`, default `auto`) states how much deliberation you are buying.
  Higher effort means more reasoning before the answer comes back — which
  costs more latency and unit cost. Choose per call: a FAQ lookup and a
  contract clause analysis should not spend the same budget.
- **`auto` delegates the choice.** With `effort: auto` (the default), the
  service reads the prompt and picks a spend level for you. Pin an explicit
  effort when your use case has its own latency or cost envelope.
- **Budget caps are advisory.** `max_latency_ms` and `max_estimated_cost`
  state your envelope; the service logs them and honors them on a best-effort
  basis. Set them to communicate intent — do not rely on them as hard
  enforcement yet.
- **Steering is per call.** `system` and `developer` messages shape how the
  answer is written (format, role, constraints). Multiple `user` messages are
  concatenated into one prompt.
- **`deliberation` is telemetry, not input.** The response carries a block
  recording what the request cost — effort honored, latency, estimated unit
  cost, plus internal quality signals. Read the cost fields; treat the rest
  as opaque telemetry and never feed it back as input.
- **Text only, for now.** One message is capped at 32,768 characters, and
  streaming is not yet supported — omit `stream`.

## The one call

A steered question with an explicit effort budget and your cost envelope:

```bash verify=auto
curl -sf --max-time 280 -X POST "${CHLOE_URL}/v3/chloe/chat/completions" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "chloe",
    "effort": "fast",
    "max_latency_ms": 8000,
    "max_estimated_cost": 0.02,
    "messages": [
      { "role": "system", "content": "You are a concise analyst." },
      { "role": "user", "content": "In one sentence: why do small teams ship faster?" }
    ]
  }' | python3 -c '
import json, sys
d = json.load(sys.stdin)
print("answer:", d["choices"][0]["message"]["content"])
tel = d["deliberation"]
print("effort honored:", tel["effort"],
      "| latency_ms:", tel["latency_ms"],
      "| est. cost:", tel["estimated_cost"])'
```

**Expected response** — `200` with an OpenAI-shaped body plus the telemetry
extension (internal telemetry fields elided here — read the cost fields, treat
the rest as opaque):

```json
{
  "id": "chatcmpl-…",
  "object": "chat.completion",
  "created": 1790257916,
  "model": "chloe",
  "choices": [
    { "index": 0,
      "message": { "role": "assistant", "content": "Small teams ship faster because …" },
      "finish_reason": "stop" }
  ],
  "usage": { "prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0 },
  "deliberation": { "effort": "fast", "latency_ms": 48903, "estimated_cost": 2.0 }
}
```

Three fields to read correctly: the answer is
`choices[0].message.content`; `model` simply echoes your request value
(`chloe` when omitted); and `usage` is a placeholder (all zeros) reserved for
a future metering release — the cost signal today is `deliberation`'s
`latency_ms` and `estimated_cost`.

> [!NOTE]
> Latency scales with effort. On the local stack the `fast` path answers in
> under a minute on a small local model; deeper effort levels deliberate
> longer, and on fleets with mature cloud models the same calls answer faster
> and at finer cost granularity.

## The effort knob

| Effort | What you are buying |
|---|---|
| `fast` | Single pass — minimum latency and cost. No escalation. |
| `balanced` | Deeper check before the answer ships; early exit when the answer is already solid. |
| `deep` | Deliberate single-domain precision. |
| `maximum` | The deepest deliberation the fleet can assemble for one call. |
| `auto` | The service picks a spend level from the prompt (the default). |

Start at `auto`, watch `deliberation.latency_ms` and `deliberation.estimated_cost`
in production traffic, and pin an explicit effort where your envelope demands
it.

## Error contract

Every validation failure is a `400` with a plain-text explanation, and a
missing or invalid JWT is a `401`:

```bash verify=auto
KEYS=(-H "x-api-key: ${INAYA_API_KEY}" -H "x-client-app-key: ${INAYA_APP_KEY}")

echo "== missing JWT =="
curl -s -o /tmp/chloe_err1.json -w "%{http_code}\n" -X POST "${CHLOE_URL}/v3/chloe/chat/completions" \
  "${KEYS[@]}" -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"hi"}]}'
cat /tmp/chloe_err1.json; echo

echo "== stream requested =="
curl -s -w "\n%{http_code}\n" -X POST "${CHLOE_URL}/v3/chloe/chat/completions" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}" -H "Content-Type: application/json" \
  -d '{"stream":true,"messages":[{"role":"user","content":"hi"}]}'

echo "== unsupported role =="
curl -s -w "\n%{http_code}\n" -X POST "${CHLOE_URL}/v3/chloe/chat/completions" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}" -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"assistant","content":"hi"}]}'

echo "== empty messages =="
curl -s -w "\n%{http_code}\n" -X POST "${CHLOE_URL}/v3/chloe/chat/completions" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}" -H "Content-Type: application/json" \
  -d '{"messages":[]}'

echo "== empty content =="
curl -s -w "\n%{http_code}\n" -X POST "${CHLOE_URL}/v3/chloe/chat/completions" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}" -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"  "}]}'

echo "== system-only messages =="
curl -s -w "\n%{http_code}\n" -X POST "${CHLOE_URL}/v3/chloe/chat/completions" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}" -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"system","content":"You are helpful."}]}'

echo "== over the 32768-character cap =="
python3 -c 'import json; print(json.dumps({"messages":[{"role":"user","content":"x"*33000}]}))' \
  | curl -s -w "\n%{http_code}\n" -X POST "${CHLOE_URL}/v3/chloe/chat/completions" \
      -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}" -H "Content-Type: application/json" -d @-
```

**Expected** — `401` with `{"error":"Missing Authorization token"}` for a
missing/invalid JWT; `400` with, respectively, `streaming is not yet
supported; omit 'stream' or set it to false`, `Unsupported role 'assistant'.
Only 'user', 'system', and 'developer' are supported.`, `messages array must
not be empty`, `message content must not be empty`, `at least one 'user'
message is required`, and `message content exceeds the 32768 character limit`.
When the model fleet is unreachable, the call fails with `503` — treat it as
retryable.

## Full working example

One script (`chloe_chat.sh`) — QA-login, then a budgeted call whose answer
and telemetry are asserted (`AUTH_URL` is the Auth Service —
`http://127.0.0.1:40004`; `CHLOE_URL` is Chloe — `http://127.0.0.1:40000`;
`TENANT_ID` is your sandbox tenant):

```bash verify=auto
#!/usr/bin/env bash
set -euo pipefail

# 0. Obtain a user JWT (QA shortcut — see G-01 for production flows)
TOKEN=$(curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"device_id\": \"g21-demo-device\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"g21-demo\"}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])')
AUTH=(-H "Authorization: Bearer ${TOKEN}"
      -H "x-api-key: ${INAYA_API_KEY}"
      -H "x-client-app-key: ${INAYA_APP_KEY}")

# 1. Budgeted reasoning call — assert shape, effort honored, and telemetry
curl -sf --max-time 280 -X POST "${CHLOE_URL}/v3/chloe/chat/completions" "${AUTH[@]}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "chloe",
    "effort": "fast",
    "max_latency_ms": 8000,
    "max_estimated_cost": 0.02,
    "messages": [
      { "role": "user", "content": "In one sentence: why do small teams ship faster?" }
    ]
  }' | python3 -c '
import json, sys
d = json.load(sys.stdin)
assert d["object"] == "chat.completion", d
assert d["choices"][0]["finish_reason"] == "stop", d
assert d["choices"][0]["message"]["content"].strip(), "empty answer"
tel = d["deliberation"]
assert tel["effort"] == "fast", tel
assert tel["latency_ms"] > 0 and tel["estimated_cost"] > 0, tel
print("chloe call: ok")
print("answer:", d["choices"][0]["message"]["content"][:80])
print("telemetry: effort=%s latency_ms=%s est_cost=%s"
      % (tel["effort"], tel["latency_ms"], tel["estimated_cost"]))'
```

## Try it live

Open the [Playground](../../playgrounds/playground.html#ai-models) →
**Our AI Models**: *Chloe* is a fully functional chat UI on this exact
endpoint — effort knob included, internals kept server-side.

## Copy agent prompt

```prompt
You are helping a developer call the Chloe reasoning endpoint (${CHLOE_URL}).
Chloe is a stateless, OpenAI-compatible reasoning surface: POST
/v3/chloe/chat/completions with header Authorization: Bearer ${TOKEN} plus the
x-api-key / x-client-app-key pair. Body: optional model label ("chloe"),
messages array (roles user, system, developer; at least one user message;
content non-empty, max 32768 chars each), optional effort budget knob
(fast|balanced|deep|maximum|auto, default auto), optional advisory
max_latency_ms / max_estimated_cost caps. No streaming — omit stream. Steps:
(1) POST one user question with effort "fast" and report
choices[0].message.content plus deliberation.effort, deliberation.latency_ms,
and deliberation.estimated_cost. (2) Explain to the developer how to choose
an effort level for latency-sensitive vs. quality-sensitive calls. Treat the
deliberation block as telemetry, never as input. Errors: 401
{"error":"Missing Authorization token"}; 400 for unsupported roles, empty
messages/content, or missing user message; 503 when the model fleet is
unreachable. Never log the JWT.
```

## API reference

- [Chloe — Swagger UI](https://chloe-service-sandbox.inayalabs.com/v3/chloe/swagger-ui/) · [Scalar](https://chloe-service-sandbox.inayalabs.com/v3/chloe/scalar) — the reasoning endpoint

## Next journeys

- **Choosing Evergreen vs Bella vs Chloe** (G-13) — where this surface fits among the platform's three AI surfaces.
- **Chatting with Bella** (G-20) — the persona surface: a pre-tuned assistant instead of a budget knob.
- **Your first AI chat session** (G-10) — full server-side conversation memory via Evergreen sessions, when one-shot is not enough.

### G-30 · Managing users & profiles
URL: /docs/guides/managing-users-profiles.html

> Read your own profile from Users Query, flesh it out with Users Command (get-modify-put), discover users via search and batch reads, and remove a user — then watch it be born again at the next sign-in.

> **Journey.** As a developer, I want my app to let people see and edit their
> profile, find other users, and close their account — and I want to understand
> exactly what happens to a user record over its whole lifetime.

The Users platform is a CQRS pair: **Users Command** (USC) owns every write,
**Users Query** (USQ) owns every read. This guide walks the full lifecycle of a
user record — born at first sign-in, fleshed out with profile data, read back
in several ways, removed, and born again — verified against the local stack
(USQ `:40170`, USC `:40171`); the shared dev subdomains behave identically.

## Prerequisites

- Both Users services are reachable (local stack USQ `:40170` / USC `:40171`,
  or the shared dev subdomains).
- Your tenant key pair in `INAYA_API_KEY` / `INAYA_APP_KEY`, **and** a user JWT
  in `TOKEN` (obtain one via guide G-01; the full example below fetches one via
  the QA sign-in shortcut).
- `curl` plus `python3` for the JSON-editing step (any HTTP client works; the
  same calls translate directly).

## Concepts (only what this journey needs)

- **Users are born at sign-in.** There is no "create user" endpoint. The first
  time an identity signs in, the platform creates a bare user record: the
  username becomes the initial `user_name` / `display_name`, email/phone and
  the remaining profile fields are empty, the role defaults to `Subscriber`,
  and `profile_json` is unset. Delete that record and the next sign-in simply
  creates a fresh one.
- **The JWT `sub` claim *is* the user id.** Every authenticated request knows
  who it belongs to; extract `sub` from the token and that is your `/v3/users/{id}`.
- **CQRS split.** All writes go to Users Command (`PUT`, `DELETE` on
  `/v3/users/{id}`); all reads come from Users Query (`GET` on
  `/v3/users/{id}`, `POST /v3/search`, `POST /v3/users/batch`, follower lists).
- **Writes are self-only.** A JWT may only `PUT` or `DELETE` the user whose id
  equals its `sub`; touching any other id returns `401`.
- **Update = get, modify, put the whole DTO.** `PUT` expects a complete
  `UserDto`. Partial bodies fail with `422` ("missing field …"). And although
  you must *send* the whole DTO, only profile fields are actually written:
  names, `display_name`, `bio`, `avatar_url`, `gender`, `mantra`,
  `location_display_name`, `social_handles`, `websites`, `profile_json`, and
  the sentiment fields. `email_address`, `phone_number`, `user_name`, and
  `roles` are ignored by this call — email and phone have their own dedicated
  storage paths, roles are managed by the platform.
- **`user_name` mirrors `display_name`.** Setting a display name makes
  `user_name` read back with the same value; clients should treat display name
  as the source of truth.
- **`profile_json` is your free-form corner.** Any JSON object your app needs
  (preferences, app-specific flags, onboarding state) rides along on the user
  record.

## Step 1 — Who am I?

Decode your user id from the JWT's `sub` claim, then read the record from
Users Query:

```bash verify=auto
USER_ID=$(python3 - <<'EOF'
import base64, json, os
segment = os.environ["TOKEN"].split(".")[1]
segment += "=" * (-len(segment) % 4)
print(json.loads(base64.urlsafe_b64decode(segment))["sub"])
EOF
)

curl -s "${USQ_URL}/v3/users/${USER_ID}" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}"
```

**Expected response** — `200` with the full `UserDto`. A freshly-born user
reads back with the username as both `user_name` and `display_name`, empty
strings for `first_name` / `last_name` / `email_address` / `phone_number`,
`roles: ["Subscriber"]`, and `profile_json: null`.

## Step 2 — Flesh out the profile

The reliable update pattern is **read the current DTO, change the fields you
own, put it back whole**. Here we set names, display name, bio, location, and a
free-form `profile_json`, then read back to prove it:

```bash verify=auto
USER_ID=$(python3 - <<'EOF'
import base64, json, os
segment = os.environ["TOKEN"].split(".")[1]
segment += "=" * (-len(segment) % 4)
print(json.loads(base64.urlsafe_b64decode(segment))["sub"])
EOF
)
AUTH=(-H "Authorization: Bearer ${TOKEN}"
      -H "x-api-key: ${INAYA_API_KEY}"
      -H "x-client-app-key: ${INAYA_APP_KEY}")

curl -s "${USQ_URL}/v3/users/${USER_ID}" "${AUTH[@]}" > /tmp/me.json

python3 - <<'EOF'
import json
me = json.load(open("/tmp/me.json"))
me["first_name"] = "Dana"
me["last_name"] = "Verifier"
me["display_name"] = "Dana V."
me["bio"] = "Owns the demo farm account."
me["location_display_name"] = "Demo Valley"
me["profile_json"] = {"favorite_animal": "chickens", "onboarded": True}
json.dump(me, open("/tmp/me-updated.json", "w"))
EOF

curl -s -X PUT "${USC_URL}/v3/users/${USER_ID}" "${AUTH[@]}" \
  -H "Content-Type: application/json" \
  --data-binary @/tmp/me-updated.json

echo
curl -s "${USQ_URL}/v3/users/${USER_ID}" "${AUTH[@]}" \
  | python3 -c 'import json,sys; u=json.load(sys.stdin); print(json.dumps({k: u[k] for k in ("display_name","user_name","bio","location_display_name","profile_json")}, indent=2))'
```

**Expected response** — the `PUT` returns `200` with the updated `UserDto`,
and the follow-up read shows `display_name: "Dana V."`, `user_name: "Dana V."`
(mirrored), the bio and location you set, and your `profile_json` object
intact. Fields you did not touch keep their previous values; `email_address`,
`phone_number`, `user_name`, and `roles` come back exactly as they were
regardless of what you sent.

## Step 3 — Find users (search & batch)

Users Query offers two discovery reads. `POST /v3/search` matches a criteria
object — each of `name` (first/last/display/nick), `email`, and
`phone_number` is optional but length-validated — and returns a plain JSON
array (no paging envelope). `POST /v3/users/batch` resolves a list of ids to
full records in one round trip:

```bash verify=auto
curl -s -X POST "${USQ_URL}/v3/search" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"search_criteria":[{"name":"Dana"}]}'
echo
USER_ID=$(python3 - <<'EOF'
import base64, json, os
segment = os.environ["TOKEN"].split(".")[1]
segment += "=" * (-len(segment) % 4)
print(json.loads(base64.urlsafe_b64decode(segment))["sub"])
EOF
)
curl -s -X POST "${USQ_URL}/v3/users/batch" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Content-Type: application/json" \
  -d "{\"ids\":[\"${USER_ID}\"]}"
```

**Expected response** — both calls return `200` with a raw JSON array of
matching `UserDto`s. A name that matches nothing returns `[]`, not an error.
(`page` / `page_size` query params are accepted by search when you need to
page large result sets.)

## Step 4 — Remove a user (and watch it be reborn)

`DELETE` is self-only and idempotent: it returns `204` even when repeated.
Afterwards the record reads `404` — and the *next sign-in creates it anew*,
empty, exactly like the very first sign-in:

```bash verify=auto
USER_ID=$(python3 - <<'EOF'
import base64, json, os
segment = os.environ["TOKEN"].split(".")[1]
segment += "=" * (-len(segment) % 4)
print(json.loads(base64.urlsafe_b64decode(segment))["sub"])
EOF
)
AUTH=(-H "Authorization: Bearer ${TOKEN}"
      -H "x-api-key: ${INAYA_API_KEY}"
      -H "x-client-app-key: ${INAYA_APP_KEY}")

curl -s -o /dev/null -w "DELETE -> HTTP %{http_code}\n" \
  -X DELETE "${USC_URL}/v3/users/${USER_ID}" "${AUTH[@]}"

curl -s -o /dev/null -w "GET after delete -> HTTP %{http_code}\n" \
  "${USQ_URL}/v3/users/${USER_ID}" "${AUTH[@]}"

# Same identity, next sign-in: a brand-new record is born at a NEW id.
TOKEN=$(curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"device_id\": \"g30-rebirth\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"g30-verify\"}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])')
export TOKEN

NEW_ID=$(python3 - <<'EOF'
import base64, json, os
segment = os.environ["TOKEN"].split(".")[1]
segment += "=" * (-len(segment) % 4)
print(json.loads(base64.urlsafe_b64decode(segment))["sub"])
EOF
)

curl -s "${USQ_URL}/v3/users/${NEW_ID}" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  | python3 -c 'import json,sys; u=json.load(sys.stdin); print("reborn:", u["display_name"] == "g30-verify" and u["profile_json"] is None)'
```

**Expected response** — `DELETE -> HTTP 204`, `GET after delete -> HTTP 404`,
then `reborn: True`. Because removal is a hard delete, the re-signed-in
identity gets a **new** user id (`NEW_ID` ≠ the deleted `USER_ID`) with a
fresh, empty profile — the username appears as the initial
`user_name` / `display_name`. (Production sign-in flows key off the verified
identity, so the profile you rebuild is the one your app sees; the QA shortcut
keys off the username the same way.) The username here (`g30-verify`) is
arbitrary — it only needs a distinct value per run.

## Error contract

Two failure modes are worth asserting in your client. Note the second one
deliberately sends a *well-formed* DTO: the body must deserialize into a
complete `UserDto` before the handler runs, so a malformed body aimed at
someone else's id fails with `422` (missing fields) before ownership is ever
considered.

```bash verify=auto
AUTH=(-H "Authorization: Bearer ${TOKEN}"
      -H "x-api-key: ${INAYA_API_KEY}"
      -H "x-client-app-key: ${INAYA_APP_KEY}")

# Unknown id -> 404 with a plain error body.
curl -s -w "\nHTTP %{http_code}\n" "${USQ_URL}/v3/users/no-such-user" "${AUTH[@]}"

# Writing to someone else's id -> 401 (self-only writes). The DTO is built
# locally so this block does not depend on any particular user existing.
python3 - <<'EOF'
import json
dto = {
    "id": "somebody-else", "user_name": "demo-user",
    "display_name": "demo-user", "first_name": "", "last_name": "",
    "email_address": "demo@example.com", "phone_number": "", "gender": "",
    "bio": None, "mantra": None, "avatar_url": None,
    "location_display_name": None, "middle_name": None, "nick_name": None,
    "roles": ["Subscriber"], "sentiment_label": "Neutral",
    "sentiment_score": 0.0, "social_handles": None, "websites": None,
    "profile_json": None, "translated": False,
    "birth_date": "2026-01-01T00:00:00Z",
    "created_at": "2026-01-01T00:00:00Z",
    "updated_at": "2026-01-01T00:00:00Z",
}
json.dump(dto, open("/tmp/g30-error-dto.json", "w"))
EOF
curl -s -o /dev/null -w "cross-user PUT -> HTTP %{http_code}\n" \
  -X PUT "${USC_URL}/v3/users/somebody-else" "${AUTH[@]}" \
  -H "Content-Type: application/json" \
  --data-binary @/tmp/g30-error-dto.json
```

**Expected response** — the unknown-id read returns
`{"error":"User not found"}` with `404`; the cross-user write returns `401`.

## Full working example

A complete born → flesh out → read → search → remove → reborn cycle
(`user_lifecycle.sh`) — self-contained, fetching its JWT via the QA shortcut
(local/QA environments only; production uses the G-01 flows):

```bash verify=auto
#!/usr/bin/env bash
set -euo pipefail

# 0. Obtain a user JWT (QA shortcut — see G-01 for production flows)
TOKEN=$(curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"device_id\": \"g30-demo\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"g30-demo-run\"}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])')
export TOKEN

AUTH=(-H "Authorization: Bearer ${TOKEN}"
      -H "x-api-key: ${INAYA_API_KEY}"
      -H "x-client-app-key: ${INAYA_APP_KEY}")

USER_ID=$(python3 -c 'import base64,json,os
segment = os.environ["TOKEN"].split(".")[1]
segment += "=" * (-len(segment) % 4)
print(json.loads(base64.urlsafe_b64decode(segment))["sub"])')
export USER_ID

# 1. Born at sign-in: read the fresh, empty record
curl -sf "${USQ_URL}/v3/users/${USER_ID}" "${AUTH[@]}" > /tmp/g30-me.json
echo "born: id=${USER_ID}"

# 2. Flesh out (get -> modify -> put the whole DTO)
python3 - <<'EOF'
import json
me = json.load(open("/tmp/g30-me.json"))
me["first_name"] = "Dana"
me["last_name"] = "Verifier"
me["display_name"] = "Dana V."
me["bio"] = "Owns the demo farm account."
me["location_display_name"] = "Demo Valley"
me["profile_json"] = {"favorite_animal": "cows", "onboarded": True}
json.dump(me, open("/tmp/g30-me-updated.json", "w"))
EOF
curl -sf -X PUT "${USC_URL}/v3/users/${USER_ID}" "${AUTH[@]}" \
  -H "Content-Type: application/json" \
  --data-binary @/tmp/g30-me-updated.json > /dev/null
echo "profile updated"

# 3. Read back via search and batch
curl -sf -X POST "${USQ_URL}/v3/search" "${AUTH[@]}" \
  -H "Content-Type: application/json" \
  -d '{"search_criteria":[{"name":"Dana"}]}' | grep -q '"Dana V."' \
  && echo "search found the profile"
curl -sf -X POST "${USQ_URL}/v3/users/batch" "${AUTH[@]}" \
  -H "Content-Type: application/json" \
  -d "{\"ids\":[\"${USER_ID}\"]}" | grep -q "${USER_ID}" \
  && echo "batch resolved the id"

# 4. Remove (idempotent 204), then confirm 404
code=$(curl -s -o /dev/null -w "%{http_code}" -X DELETE \
  "${USC_URL}/v3/users/${USER_ID}" "${AUTH[@]}")
[[ "$code" == "204" ]] && echo "deleted (204)"
code=$(curl -s -o /dev/null -w "%{http_code}" \
  "${USQ_URL}/v3/users/${USER_ID}" "${AUTH[@]}")
[[ "$code" == "404" ]] && echo "gone (404)"

# 5. Rebirth at next sign-in
curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"device_id\": \"g30-demo\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"g30-demo-run\"}" \
  > /dev/null
echo "re-signed-in; profile state resets to empty on the platform side"
```

(`AUTH_URL` is the Auth Service base URL — `http://127.0.0.1:40004` on the
local stack; `TENANT_ID` is your tenant's id; `USQ_URL` / `USC_URL` are the
Users Query / Users Command base URLs.)

## Try it live

Open the [Playground](../../playgrounds/playground.html#identity) →
**Users & Profile Matching** — the *Users* console exercises the same
get → update → search → remove cycle against the live Users services.

## Copy agent prompt

```prompt
You are working with the Inaya Labs Users platform. USQ (reads) and USC
(writes) base URLs are in USQ_URL / USC_URL; the Auth Service is at AUTH_URL;
the tenant key pair is in INAYA_API_KEY / INAYA_APP_KEY and TENANT_ID. Obtain
a user JWT via POST /v3/auth/qa/login {"device_id","locale","tenant_id",
"username"} if TOKEN is not already set; the JWT "sub" claim is the user id.
Steps: (1) GET /v3/users/{sub} from USQ and report whether profile_json is
set. (2) Update the profile: fetch the full UserDto, set display_name to
"Agent Friend", bio to "Updated by an agent", profile_json to
{"agent_visit": true}; PUT the whole DTO to USC /v3/users/{sub} (expect 200)
and confirm via a fresh GET. (3) Search via POST USQ /v3/search
{"search_criteria":[{"name":"Agent Friend"}]} and confirm the user appears in
the returned array. (4) Attempt PUT to /v3/users/somebody-else and report the
401. (5) DELETE /v3/users/{sub} (expect 204), confirm a subsequent GET is 404,
then re-login and report that the profile reads back empty. Never log the JWT;
report status codes per step.
```

## API reference

- [Users Query — Swagger UI](https://users-query-service-sandbox.inayalabs.com/v3/swagger-ui/) · [Users Command — Swagger UI](https://users-command-service-sandbox.inayalabs.com/v3/swagger-ui/)

## Next journeys

- **Federated sign-in for your app** (G-31) — list federated login methods and complete a sign-in flow end-to-end.
- **Finding matches between profiles** (G-32) — collections, profile indexing, and threshold-based search.
- **Authentication & sessions** (G-01) — the token lifecycle that every call in this guide depends on.

### G-31 · Federated sign-in for your app
URL: /docs/guides/federated-sign-in.html

> Ask the Auth Service which sign-in methods your app may offer, complete a federated Google/Apple sign-in end-to-end, start the passkey ceremony, and rotate or revoke the session that comes back.

> **Journey.** As a developer, I want my users to sign in with the identity
> providers they already trust — Apple, Google, or a device passkey — and I
> want to discover which of those methods my app can offer, for which users,
> without hard-coding anything.

The Auth Service is the platform's only front door for identity. It has no
username/password database: production sign-in is **federated** (Apple or
Google ID tokens) or **passkey** (WebAuthn). This guide covers the discovery
call that tells your app which methods to render, a complete federated sign-in
end-to-end, the passkey ceremony's shape, and the session lifecycle (refresh
rotation, logout) that every sign-in returns into. Verified against the local
stack (auth on `:40004`); the shared dev subdomain behaves identically.

## Prerequisites

- The Auth Service is reachable (local stack `:40004`, or the shared dev
  subdomain).
- Your tenant key pair in `INAYA_API_KEY` / `INAYA_APP_KEY` and your tenant id
  in `TENANT_ID`. Sign-in endpoints sit in the *vendor-auth* tier: the key
  pair is required, but **no user JWT exists yet** — that is what sign-in
  issues.
- `curl` plus `python3` for JSON assertions. Snippets also show the response
  shapes your client code should expect.

## Concepts (only what this journey needs)

- **Three auth tiers, one router.** *Public* routes (`/v3/health`,
  `/v3/auth/login_methods`, docs) need no credentials. *Vendor-auth* routes
  (federated login, passkey begin/finish, refresh, and the lower-environment
  QA login) need the tenant key pair. *JWT routes* (`/v3/auth/logout`) need
  the key pair **plus** a user JWT — they are only callable once sign-in has
  succeeded.
- **Discovery before rendering.** Call `POST /v3/auth/login_methods` with the
  user's locale. It returns `all_methods` (the full catalog the platform
  understands) and `regional_methods` (what this locale may actually offer —
  today every region resolves to Apple sign-in, and a locale without a region
  suffix returns an empty list). Render the intersection with what your app
  supports.
- **Federated login exchanges a provider token for a platform session.**
  `POST /v3/auth/federated/login` takes the provider (`Apple` or `Google`) and
  the provider-issued `id_token`; the service verifies it with the provider,
  finds or creates the user, and returns `{refresh_token, token, user}` — the
  same response shape every sign-in path returns. A fresh user is born with
  the default `Subscriber` role (see G-30 for the full user lifecycle).
- **The local environment has a development bypass.** When the service runs
  with `INAYA_LABS_ENVIRONMENT=local`, the `id_token` field is instead read as
  a **user id**: an existing user is signed in directly, an unknown id mints a
  fresh user. This lets you exercise the entire federated flow without real
  Apple/Google tokens. It never activates outside `local`.
- **Passkeys use the WebAuthn two-step ceremony.** `register/begin` issues a
  creation challenge, the device authenticator signs it, `register/finish`
  stores the credential; `login/begin` + `login/finish` mirror that for
  assertions. Your client passes `options` to
  `navigator.credentials.create()` / `.get()` and returns the signed
  credential to the matching `finish` endpoint.
- **Sessions rotate and can be revoked.** The `refresh_token` rotates on every
  use — reuse of an old one is treated as theft and revokes the session
  (`403`). `PUT /v3/auth/logout` revokes the caller's JWT. Full token
  lifecycle detail lives in G-01.

## Step 1 — Discover the methods your app may offer

`login_methods` is public — no key pair, no JWT — so your sign-in screen can
call it before anything exists:

```bash verify=auto
echo "-- locale with a region suffix (en-US) --"
curl -s -X POST "${AUTH_URL}/v3/auth/login_methods" \
  -H "Content-Type: application/json" \
  -d '{"locale":"en-US"}'
echo
echo "-- locale without a region (en) --"
curl -s -X POST "${AUTH_URL}/v3/auth/login_methods" \
  -H "Content-Type: application/json" \
  -d '{"locale":"en"}'
echo
echo "-- missing locale -> 422 --"
curl -s -o /dev/null -w "HTTP %{http_code}\n" -X POST \
  "${AUTH_URL}/v3/auth/login_methods" \
  -H "Content-Type: application/json" -d '{}'
```

**Expected response** — `200` with
`{"all_methods":["FederatedApple","FederatedGoogle","Passkey"],"regional_methods":[...]}`.
`regional_methods` is `["FederatedApple"]` for any region-suffixed locale and
`[]` when the locale carries no region. A body without `locale` is rejected
`422` at deserialization.

## Step 2 — Federated sign-in, end to end

In `local` the `id_token` field carries a **user id**, so the full flow is:
create a user (here via the QA shortcut — see G-01), then federate to that id.
The response must come back with the same user and a fresh session:

```bash verify=auto
KEYS=(-H "x-api-key: ${INAYA_API_KEY}"
      -H "x-client-app-key: ${INAYA_APP_KEY}")

# 0. Create a user to federate to (QA shortcut — lower environments only;
#    production supplies a real Apple/Google id_token instead).
TOKEN=$(curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" "${KEYS[@]}" \
  -H "Content-Type: application/json" \
  -d "{\"device_id\": \"g31-demo\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"g31-demo-user\"}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])')
export TOKEN
USER_ID=$(python3 - <<'EOF'
import base64, json, os
segment = os.environ["TOKEN"].split(".")[1]
segment += "=" * (-len(segment) % 4)
print(json.loads(base64.urlsafe_b64decode(segment))["sub"])
EOF
)

# 1. Federated sign-in to that user id (local bypass stands in for a real
#    provider token; provider must still be a valid enum value).
curl -s -X POST "${AUTH_URL}/v3/auth/federated/login" "${KEYS[@]}" \
  -H "Content-Type: application/json" \
  -d "{\"device_id\": \"g31-demo\", \"id_token\": \"${USER_ID}\", \"locale\": \"en-US\", \"provider\": \"Google\", \"tenant_id\": \"${TENANT_ID}\"}" \
  > /tmp/g31-fed.json

python3 - <<EOF
import json
r = json.load(open("/tmp/g31-fed.json"))
print("keys:", sorted(r.keys()))
print("same user:", r["user"]["id"] == "${USER_ID}")
print("display_name:", repr(r["user"]["display_name"]), "| roles:", r["user"]["roles"])
print("access token present:", bool(r["token"]), "| refresh token present:", bool(r["refresh_token"]))
EOF
```

**Expected response** — `200` with `token`, `refresh_token`, and a `user` whose
`id` equals the id you federated to. Two things worth knowing about the
bypass's find-or-create: passing an **unknown** id mints a *fresh* user each
call (the new record's id is random, not the id you passed, so the next call
misses again — don't loop it), and the minted user has an empty
`display_name` until your app sets one via G-30. In production the same call
with a real provider `id_token` finds or creates the user tied to that
provider identity, stably.

## Step 3 — The passkey ceremony

Passkeys are the platform's passwordless path: registration binds a device
credential to a user; login asserts it. Both are begin/finish pairs — your
client drives the authenticator between the two calls.

**Login begin** (userless / discoverable-credential mode — no email) works
today and returns the challenge your browser feeds to
`navigator.credentials.get()`:

```bash verify=auto
curl -s -X POST "${AUTH_URL}/v3/auth/pass_key/login/begin" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Content-Type: application/json" \
  -d "{\"device_id\": \"g31-demo\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\"}" \
  | python3 -c 'import json,sys; o=json.load(sys.stdin); print("challenge_id present:", bool(o["challenge_id"])); print("publicKey fields:", sorted(o["options"]["publicKey"].keys()))'
```

**Expected response** — `200` with `challenge_id` and
`options.publicKey` (a standard WebAuthn `PublicKeyCredentialRequestOptions`:
`challenge`, `rpId`, `timeout`, `userVerification`, `allowCredentials`). Your
client posts the signed assertion to `pass_key/login/finish` with the
`challenge_id`; a valid assertion returns the same
`{refresh_token, token, user}` shape as federated login. A wrong or stale
assertion returns `401`.

**Registration** begins with `pass_key/register/begin`: the service finds or
creates the account for the email, issues a creation challenge, and your
browser feeds `options` to `navigator.credentials.create()`. The signed
attestation goes to `register/finish`, which stores the credential and returns
`{"registered": true}`:

```bash verify=auto
curl -s -X POST "${AUTH_URL}/v3/auth/pass_key/register/begin" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Content-Type: application/json" \
  -d "{\"email\": \"g31-passkey@example.com\", \"platform\": \"macbook-pro-m4\", \"device_id\": \"g31-demo\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\"}" \
  | python3 -c 'import json,sys; o=json.load(sys.stdin); print("challenge_id present:", bool(o["challenge_id"])); print("publicKey fields:", sorted(o["options"]["publicKey"].keys()))'
```

**Expected response** — `200` with `challenge_id` and `options.publicKey` (a
standard WebAuthn `PublicKeyCredentialCreationOptions`: `challenge`, `rp`,
`user`, `pubKeyCredParams`, `authenticatorSelection`, `attestation`,
`extensions`, `timeout`). The platform user id is mapped to a stable UUIDv5
handle under the hood, so the same account always presents the same RP user
handle. `register/finish` posts the signed credential with the `challenge_id`
and returns `200 {"registered": true}`; a bad attestation returns `401` with a
JSON `error` body, and a validation failure returns `400` with one too.

Two behaviors to design around:

- **`login/begin` scoped to an email fails `400`** (`{"error":"No registered
  passkeys found for this email address"}`) until that email has at least one
  registered credential — standard WebAuthn semantics. The userless
  (discoverable-credential) begin above always works.
- **A passkey-provisioned account *is* found-or-created by email.** The email
  supplied to `register/begin` is persisted on the freshly provisioned account
  as an unverified lookup key (the WebAuthn credential itself is the proof of
  control; the email is only used to resolve the account on later calls).
  Repeating `register/begin` for the same address therefore reuses the same
  account and presents the same RP user handle — the account is not
  re-provisioned per call. Email-scoped `login/begin` still fails `400` until
  at least one credential's `finish` has landed for that account.

## Step 4 — The session that sign-in returns

Every successful sign-in returns the same session triple. Rotate the refresh
token to extend the session; logout to end it:

```bash verify=auto
KEYS=(-H "x-api-key: ${INAYA_API_KEY}"
      -H "x-client-app-key: ${INAYA_APP_KEY}")
RESP=$(curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" "${KEYS[@]}" \
  -H "Content-Type: application/json" \
  -d "{\"device_id\": \"g31-demo\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"g31-demo-user\"}")
TOKEN=$(printf '%s' "$RESP" | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])')
REFRESH=$(printf '%s' "$RESP" | python3 -c 'import json,sys; print(json.load(sys.stdin)["refresh_token"])')

# 1. Refresh rotates: a new access token AND a new refresh token are issued.
curl -s -X POST "${AUTH_URL}/v3/auth/refresh" "${KEYS[@]}" \
  -H "Content-Type: application/json" \
  -d "{\"device_id\": \"g31-demo\", \"refresh_token\": \"${REFRESH}\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\"}" \
  > /tmp/g31-refresh.json
python3 -c 'import json; r=json.load(open("/tmp/g31-refresh.json")); print("rotated:", bool(r["token"]) and bool(r["refresh_token"]))'

# 2. Reusing the OLD refresh token is treated as theft -> 403, session revoked.
curl -s -o /dev/null -w "reuse -> HTTP %{http_code}\n" -X POST \
  "${AUTH_URL}/v3/auth/refresh" "${KEYS[@]}" \
  -H "Content-Type: application/json" \
  -d "{\"device_id\": \"g31-demo\", \"refresh_token\": \"${REFRESH}\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\"}"

# 3. Logout revokes the caller's JWT (JWT + key pair required).
curl -s -w "\nlogout -> HTTP %{http_code}\n" -X PUT \
  "${AUTH_URL}/v3/auth/logout" "${KEYS[@]}" \
  -H "Authorization: Bearer ${TOKEN}"
```

**Expected response** — refresh returns `200` with a brand-new token pair;
reusing the superseded refresh token returns `403`
`{"error":"parent_token_expired"}`; logout returns `200`
`"Logged out successfully"`. Calling logout without a JWT returns `401`.
(Token expiry, retirement, and the revocation store are G-01 territory.)

## Full working example

A complete discover → federate → refresh → logout cycle
(`federated_signin.sh`), self-contained via the local bypass:

```bash verify=auto
#!/usr/bin/env bash
set -euo pipefail

KEYS=(-H "x-api-key: ${INAYA_API_KEY}"
      -H "x-client-app-key: ${INAYA_APP_KEY}")

# 1. Discover methods for the sign-in screen
curl -sf -X POST "${AUTH_URL}/v3/auth/login_methods" \
  -H "Content-Type: application/json" -d '{"locale":"en-US"}' \
  | grep -q '"FederatedGoogle"' && echo "methods discovered"

# 2. Federated sign-in end-to-end (local bypass)
TOKEN=$(curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" "${KEYS[@]}" \
  -H "Content-Type: application/json" \
  -d "{\"device_id\": \"g31-demo\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"g31-demo-user\"}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])')
export TOKEN
USER_ID=$(python3 -c 'import base64,json,os
segment = os.environ["TOKEN"].split(".")[1]
segment += "=" * (-len(segment) % 4)
print(json.loads(base64.urlsafe_b64decode(segment))["sub"])')
export USER_ID

curl -sf -X POST "${AUTH_URL}/v3/auth/federated/login" "${KEYS[@]}" \
  -H "Content-Type: application/json" \
  -d "{\"device_id\": \"g31-demo\", \"id_token\": \"${USER_ID}\", \"locale\": \"en-US\", \"provider\": \"Apple\", \"tenant_id\": \"${TENANT_ID}\"}" \
  > /tmp/g31-fed.json
python3 -c 'import json; r=json.load(open("/tmp/g31-fed.json")); assert r["user"]["id"] == "'"${USER_ID}"'", "federated user mismatch"; print("federated sign-in ok")'

# 3. Refresh the session
REFRESH=$(python3 -c 'import json; print(json.load(open("/tmp/g31-fed.json"))["refresh_token"])')
curl -sf -X POST "${AUTH_URL}/v3/auth/refresh" "${KEYS[@]}" \
  -H "Content-Type: application/json" \
  -d "{\"device_id\": \"g31-demo\", \"refresh_token\": \"${REFRESH}\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\"}" \
  > /dev/null && echo "session refreshed"

# 4. Logout
curl -sf -X PUT "${AUTH_URL}/v3/auth/logout" "${KEYS[@]}" \
  -H "Authorization: Bearer ${TOKEN}" > /dev/null && echo "logged out"
```

(`AUTH_URL` is the Auth Service base URL — `http://127.0.0.1:40004` on the
local stack; `TENANT_ID` is your tenant's id.)

## Try it live

Open the [Playground](../../playgrounds/playground.html#identity) →
**Users & Profile Matching** — the *Auth* console lists the federated login methods
and drives sign-in/sign-out against the same live Auth Service.

## Copy agent prompt

```prompt
You are working with the Inaya Labs Auth Service at AUTH_URL; the tenant key
pair is in INAYA_API_KEY / INAYA_APP_KEY and TENANT_ID. The environment is
local, so POST /v3/auth/qa/login {"device_id","locale","tenant_id",
"username"} issues a session for any username (lower environments only). Steps:
(1) POST /v3/auth/login_methods {"locale":"en-US"} (no auth needed) and report
all_methods vs regional_methods. (2) Obtain a user id via the QA login, then
complete POST /v3/auth/federated/login with that id as id_token, provider
"Google"; expect 200 with {refresh_token, token, user} where user.id matches.
(3) Start POST /v3/auth/pass_key/login/begin without an email and report the
publicKey options fields. (4) Rotate the session via POST /v3/auth/refresh
{"device_id","refresh_token","locale","tenant_id"} and confirm a new pair;
then reuse the old refresh token and report the 403. (5) PUT
/v3/auth/logout with the access token and expect 200; report the 401 without
it. Never log JWTs or refresh tokens; report status codes per step.
```

## API reference

- [Auth Service — Swagger UI](https://auth-service-sandbox.inayalabs.com/v3/auth/swagger-ui/) · [Scalar](https://auth-service-sandbox.inayalabs.com/v3/auth/scalar)

## Next journeys

- **Finding matches between profiles** (G-32) — turn the users you just signed in into matchable profiles.
- **Managing users & profiles** (G-30) — flesh out the user record every sign-in creates.
- **Authentication & sessions** (G-01) — the full token lifecycle: expiry, retirement, revocation store.

### G-32 · Finding matches between profiles
URL: /docs/guides/finding-matches-profiles.html

> Create a matchable collection, index JSON profiles into it, and rank them against a query with top-K cosine search — then filter by a similarity threshold and clean up.

> **Journey.** As a developer, I want my app to match people or things to each
> other — caretakers to households, mentors to mentees, applicants to openings —
> by indexing each side as a JSON profile and ranking them against a query,
> without running my own vector database.

The Profile Matching service turns arbitrary JSON payloads into ranked
similarity searches. You create a **collection** (a named vector space with a
vectorizer configuration), **index** a profile — any JSON object — into it, and
then **search** the collection by posting another JSON object as the query.
Every profile is scored for cosine similarity against the query and the top-K
come back ranked, each with its similarity score so your app can apply its own
threshold. This guide walks create → index → search → threshold → cleanup,
verified against the local stack (profile-matching `:45093`); the shared dev
subdomain behaves identically.

## Prerequisites

- Profile Matching is reachable (local stack `:45093`, or the shared dev
  subdomain).
- Your tenant key pair in `INAYA_API_KEY` / `INAYA_APP_KEY`, **and** a user JWT
  in `TOKEN` (obtain one via guide G-01; the full example below fetches one via
  the QA sign-in shortcut). The `POST` routes require the key pair + JWT; only
  `/v1/health` is public.
- `curl` plus `python3` for the JSON-handling steps (any HTTP client works; the
  same calls translate directly).

## Concepts (only what this journey needs)

- **Collection = a named match space.** A collection carries the vectorizer
  settings every profile indexed into it will use: `vectorizer_kind`
  (`simple_json_v1` is the only kind today) and `embedding_dim` (default
  `256`). Pick one collection per matching domain — "caretakers", "mentors",
  "listings" — not one for the whole app.
- **Profile = any JSON object.** The payload is flattened into
  `path=value` features and feature-hashed into the collection's embedding at
  index time. Shared `key=value` pairs are what make two profiles similar;
  numbers contribute by key path (so the same numeric key reinforces), booleans
  by `true`/`false`, arrays by position (keep array shapes consistent across
  profiles or indexes misalign), and `null` contributes nothing.
- **Search ranks, it does not threshold.** `POST /v1/search` returns the top-K
  results sorted by descending cosine similarity, each with its `similarity`
  score, plus `total_candidates` (how many profiles were scored). There is no
  server-side threshold parameter — apply your own cutoff on the score
  (`similarity >= 0.5`, say) once results are back.
- **`external_key` is your correlation handle.** Indexing accepts an optional
  `external_key` — your app's own id for the entity (a user id, a listing id).
  It is echoed back in every result so you can join matches to your own records.
- **No update endpoint — re-index instead.** To change a profile, index a new
  one and `DELETE` the old; there is intentionally no `PUT`. `DELETE` is
  idempotent (`204` even for an already-gone id).
- **Reads are scoped to what you keep.** There is no "list collections" or
  "list profiles" route — your app remembers the ids it created (or stores
  them via `external_key`). Collections cannot be deleted in V1; only their
  profiles can. Treat collection creation as provisioning, not churn.

## Step 1 — Create a collection

One `POST` provisions the match space. The response echoes the id every later
call needs:

```bash verify=auto
curl -s -X POST "${PM_URL}/v1/collections" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"name": "g32-caretakers", "embedding_dim": 256}' \
  | tee /tmp/g32-collection.json \
  | python3 -c 'import json,sys; c=json.load(sys.stdin); print("collection:", c["id"], c["vectorizer_kind"], c["embedding_dim"])'
```

**Expected response** — `201` with the collection: `id`, `name`,
`vectorizer_kind: "simple_json_v1"`, `embedding_dim`, `created_at`. Both fields
are optional; omitting them gives the defaults (`simple_json_v1` / `256`).

## Step 2 — Index profiles

Index each candidate as a JSON payload with your own `external_key`. Here we
seed three caretakers — two shepherds and an accountant — so the search in
Step 3 has something meaningful to rank:

```bash verify=auto
COLLECTION_ID=$(python3 -c 'import json; print(json.load(open("/tmp/g32-collection.json"))["id"])')
AUTH=(-H "Authorization: Bearer ${TOKEN}"
      -H "x-api-key: ${INAYA_API_KEY}"
      -H "x-client-app-key: ${INAYA_APP_KEY}")

for profile in \
  '{"external_key": "caretaker-amina", "payload": {"role": "shepherd", "experience_years": 5, "looking_for": "pasture-work"}}' \
  '{"external_key": "caretaker-bo",    "payload": {"role": "shepherd", "experience_years": 3, "looking_for": "pasture-work"}}' \
  '{"external_key": "caretaker-cy",    "payload": {"role": "accountant", "experience_years": 10, "looking_for": "desk-work"}}'
do
  curl -s -o /dev/null -w "indexed -> HTTP %{http_code}\n" \
    -X POST "${PM_URL}/v1/collections/${COLLECTION_ID}/profiles" "${AUTH[@]}" \
    -H "Content-Type: application/json" -d "$profile"
done
```

**Expected response** — each call returns `201` with the stored profile: a
server-generated `id`, the `collection_id`, your `external_key`, the `payload`
as stored, and timestamps. Keep the profile `id` (or your `external_key`) —
that is how you delete the profile later.

## Step 3 — Search and rank

Post a query payload in the same shape as your profiles. The service vectorizes
it exactly like an indexed profile and scores every profile in the collection;
the top-K come back ranked with their similarity:

```bash verify=auto
COLLECTION_ID=$(python3 -c 'import json; print(json.load(open("/tmp/g32-collection.json"))["id"])')

curl -s -X POST "${PM_URL}/v1/search" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Content-Type: application/json" \
  -d "{\"collection_id\": \"${COLLECTION_ID}\", \"query\": {\"role\": \"shepherd\", \"looking_for\": \"pasture-work\"}, \"top_k\": 3}" \
  | python3 -c 'import json,sys
r = json.load(sys.stdin)
print("candidates:", r["total_candidates"])
for x in r["results"]:
    print("%s: %.3f" % (x["profile"]["external_key"], x["similarity"]))'
```

**Expected response** — `200` with `total_candidates: 3` and the two shepherds
ranked above the accountant (`caretaker-amina` and `caretaker-bo` both share
the query's `role` and `looking_for` features; the accountant shares none and
scores `0.0`). A profile identical to the query scores `1.0`; completely
disjoint payloads score `0.0`. `top_k` defaults to `5` when omitted.

## Step 4 — Apply your threshold

The API stops at ranking; the cutoff is your product decision. Fetch the top-K
you can afford, then keep only what clears your bar — here `0.2`:

```bash verify=auto
COLLECTION_ID=$(python3 -c 'import json; print(json.load(open("/tmp/g32-collection.json"))["id"])')

curl -s -X POST "${PM_URL}/v1/search" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Content-Type: application/json" \
  -d "{\"collection_id\": \"${COLLECTION_ID}\", \"query\": {\"role\": \"shepherd\", \"looking_for\": \"pasture-work\"}, \"top_k\": 10}" \
  | python3 -c 'import json,sys
r = json.load(sys.stdin)
THRESHOLD = 0.2
matches = [x["profile"]["external_key"] for x in r["results"] if x["similarity"] >= THRESHOLD]
print("matched:", matches)'
```

**Expected response** — `matched: ['caretaker-amina', 'caretaker-bo']` (both
shepherds clear 0.2; the accountant at 0.0 does not). Tune `THRESHOLD` to your
domain: with the V1 vectorizer, well-matched categorical profiles land around
`0.3–0.7`, partial overlaps lower, disjoint payloads at `0.0`. Validate against
your own data before shipping a cutoff.

> **NOTE:** `total_candidates` tells you how many profiles were actually
> scored — if it approaches `top_k`, raise `top_k` (or accept you are only
> seeing the best of a small pool).

## Step 5 — Remove a profile (and why re-index beats update)

`DELETE /v1/profiles/{id}` removes a single profile and is idempotent — a
repeat delete (or an unknown id) still returns `204`. To "edit" a profile,
index the replacement and delete the original:

```bash verify=auto
COLLECTION_ID=$(python3 -c 'import json; print(json.load(open("/tmp/g32-collection.json"))["id"])')
AUTH=(-H "Authorization: Bearer ${TOKEN}"
      -H "x-api-key: ${INAYA_API_KEY}"
      -H "x-client-app-key: ${INAYA_APP_KEY}")

# Bo gained experience: index the updated profile, then drop the stale one.
NEW_ID=$(curl -s -X POST "${PM_URL}/v1/collections/${COLLECTION_ID}/profiles" "${AUTH[@]}" \
  -H "Content-Type: application/json" \
  -d '{"external_key": "caretaker-bo", "payload": {"role": "shepherd", "experience_years": 6, "looking_for": "pasture-work"}}' \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["id"])')
echo "re-indexed as ${NEW_ID}"

curl -s -o /dev/null -w "delete stale -> HTTP %{http_code}\n" \
  -X DELETE "${PM_URL}/v1/profiles/${NEW_ID}" "${AUTH[@]}"
curl -s -o /dev/null -w "delete again (idempotent) -> HTTP %{http_code}\n" \
  -X DELETE "${PM_URL}/v1/profiles/${NEW_ID}" "${AUTH[@]}"
```

**Expected response** — `re-indexed as <new id>`, then `delete stale -> HTTP
204` and `delete again (idempotent) -> HTTP 204`. (We delete the fresh copy so
the demo data stays stable for re-runs — in a real app you would delete the
*old* profile id here and keep the new one.)

## Error contract

Three failures worth asserting in your client: a malformed collection body, a
nonexistent collection, and a malformed profile id.

```bash verify=auto
AUTH=(-H "Authorization: Bearer ${TOKEN}"
      -H "x-api-key: ${INAYA_API_KEY}"
      -H "x-client-app-key: ${INAYA_APP_KEY}")

# embedding_dim must be positive -> 400 with a JSON error body.
curl -s -w "\ncreate bad dim -> HTTP %{http_code}\n" \
  -X POST "${PM_URL}/v1/collections" "${AUTH[@]}" \
  -H "Content-Type: application/json" \
  -d '{"name": "g32-bad", "embedding_dim": 0}'

# Unknown collection id -> 404 on search.
curl -s -w "\nsearch unknown collection -> HTTP %{http_code}\n" \
  -X POST "${PM_URL}/v1/search" "${AUTH[@]}" \
  -H "Content-Type: application/json" \
  -d '{"collection_id": "00000000-0000-0000-0000-000000000000", "query": {"role": "shepherd"}}'

# Malformed profile id -> 400 (ids must be UUIDs).
curl -s -w "\ndelete malformed id -> HTTP %{http_code}\n" \
  -X DELETE "${PM_URL}/v1/profiles/not-a-uuid" "${AUTH[@]}"
```

**Expected response** — `create bad dim -> HTTP 400` with
`{"error":"Invalid input: embedding_dim must be positive"}`; `search unknown
collection -> HTTP 404` with `{"error":"Collection not found"}`; `delete
malformed id -> HTTP 400` with `{"error":"Invalid profile id"}`.

## Full working example

A complete create → index → search → threshold → cleanup cycle
(`profile_matching.sh`) — self-contained, fetching its JWT via the QA shortcut
(local/QA environments only; production uses the G-01 flows):

```bash verify=auto
#!/usr/bin/env bash
set -euo pipefail

# 0. Obtain a user JWT (QA shortcut — see G-01 for production flows)
TOKEN=$(curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"device_id\": \"g32-demo\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"g32-demo-run\"}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])')
export TOKEN

AUTH=(-H "Authorization: Bearer ${TOKEN}"
      -H "x-api-key: ${INAYA_API_KEY}"
      -H "x-client-app-key: ${INAYA_APP_KEY}")

# 1. Create the collection
COLLECTION_ID=$(curl -sf -X POST "${PM_URL}/v1/collections" "${AUTH[@]}" \
  -H "Content-Type: application/json" \
  -d '{"name": "g32-demo-caretakers", "embedding_dim": 256}' \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["id"])')
echo "collection: ${COLLECTION_ID}"

# 2. Index profiles
for profile in \
  '{"external_key": "caretaker-amina", "payload": {"role": "shepherd", "experience_years": 5, "looking_for": "pasture-work"}}' \
  '{"external_key": "caretaker-bo",    "payload": {"role": "shepherd", "experience_years": 3, "looking_for": "pasture-work"}}' \
  '{"external_key": "caretaker-cy",    "payload": {"role": "accountant", "experience_years": 10, "looking_for": "desk-work"}}'
do
  curl -sf -o /dev/null -X POST "${PM_URL}/v1/collections/${COLLECTION_ID}/profiles" \
    "${AUTH[@]}" -H "Content-Type: application/json" -d "$profile"
done
echo "3 profiles indexed"

# 3. Search and apply a threshold
curl -sf -X POST "${PM_URL}/v1/search" "${AUTH[@]}" \
  -H "Content-Type: application/json" \
  -d "{\"collection_id\": \"${COLLECTION_ID}\", \"query\": {\"role\": \"shepherd\", \"looking_for\": \"pasture-work\"}, \"top_k\": 10}" \
  | python3 -c 'import json,sys
r = json.load(sys.stdin)
keys = [x["profile"]["external_key"] for x in r["results"] if x["similarity"] >= 0.2]
assert set(keys) == {"caretaker-amina", "caretaker-bo"}, keys
print("matched:", keys)'

# 4. Cleanup: delete every profile we created via the collection-wide
#    ranking (search always scores every profile, so top_k >= count
#    returns them all regardless of the query shape).
IDS=$(curl -sf -X POST "${PM_URL}/v1/search" "${AUTH[@]}" \
  -H "Content-Type: application/json" \
  -d "{\"collection_id\": \"${COLLECTION_ID}\", \"query\": {\"role\": \"shepherd\"}, \"top_k\": 100}" \
  | python3 -c 'import json,sys
r = json.load(sys.stdin)
print(" ".join(x["profile"]["id"] for x in r["results"]))')
for id in ${IDS}; do
  curl -sf -o /dev/null -X DELETE "${PM_URL}/v1/profiles/${id}" "${AUTH[@]}"
done
echo "cleaned up profiles; the collection itself remains (V1 has no delete-collection)"
```

(`AUTH_URL` is the Auth Service base URL — `http://127.0.0.1:40004` on the
local stack; `PM_URL` is the Profile Matching base URL — `http://127.0.0.1:45093`
locally; `TENANT_ID` is your tenant's id.)

## Try it live

Open the [Playground](../../playgrounds/playground.html#identity) →
**Users & Profile Matching** — the *Profile Matching* console exercises the same
create → index → match cycle against the live service.

## Copy agent prompt

```prompt
You are working with the Inaya Labs Profile Matching service. The base URL is
in PM_URL; the tenant key pair is in INAYA_API_KEY / INAYA_APP_KEY and
TENANT_ID; the Auth Service is at AUTH_URL. Obtain a user JWT via POST
/v3/auth/qa/login {"device_id","locale","tenant_id","username"} if TOKEN is
not already set. Steps: (1) POST /v1/collections {"name":"agent-caretakers",
"embedding_dim":256} (expect 201) and remember the id. (2) Index three
profiles into it via POST /v1/collections/{id}/profiles: caretaker-amina
{"role":"shepherd","experience_years":5,"looking_for":"pasture-work"},
caretaker-bo {"role":"shepherd","experience_years":3,"looking_for":
"pasture-work"}, caretaker-cy {"role":"accountant","experience_years":10,
"looking_for":"desk-work"} — each with an external_key and expect 201. (3)
Search via POST /v1/search {"collection_id": id, "query": {"role":"shepherd",
"looking_for":"pasture-work"}, "top_k": 10} and report each result's
external_key and similarity; confirm both shepherds outrank the accountant and
note total_candidates. (4) Repeat the search keeping only similarity >= 0.2
and report the matched external_keys. (5) Delete one profile by id (expect
204), then delete the same id again and report that it is still 204
(idempotent). Never log the JWT; report status codes per step.
```

## API reference

- [Profile Matching — Swagger UI](https://profile-matching-service-sandbox.inayalabs.com/v1/swagger-ui/)

## Next journeys

- **Managing users & profiles** (G-30) — the user records whose ids make ideal `external_key`s.
- **Federated sign-in for your app** (G-31) — turn the users you just signed in into matchable profiles.
- **Authentication & sessions** (G-01) — the token lifecycle that every call in this guide depends on.

### G-40 · Publishing and reading a feed
URL: /docs/guides/publishing-reading-feed.html

> Post to the feed from your app, watch the moderation pipeline publish it, read and search the timeline, edit as the author, and clean up — across the Feed Command / Feed Query CQRS pair.

> **Journey.** As a developer, I want my app's users to publish posts, reply to
> each other, and read a ranked timeline with search — while the platform
> handles content moderation, hashtagging, sentiment, and expiry for me.

The Feed platform is a CQRS trio: **Feed Command** (FDC) owns every write,
**Feed Query** (FDQ) owns every read, and **Feed Workflow** (FDW) runs the
publish pipeline in the background — a civility check via Evergreen AI, then
platform hashtag and sentiment generation, then promotion to the published
timeline (posts auto-expire 30 days after publishing). This guide walks
publish → read → search → edit → react → delete, verified against the local
stack (FDC `:40167`, FDQ `:40168`); the shared dev subdomains behave
identically.

## Prerequisites

- Feed Command, Feed Query **and** Feed Workflow are reachable (local stack
  `:40167`/`:40168` + workflow running, or the shared dev subdomains). The
  workflow is what publishes posts — without it, writes succeed but reads stay
  empty.
- Evergreen AI must be able to serve the moderation model (the workflow pins a
  small local model automatically; on macOS it resolves via Ollama).
- Your tenant key pair in `INAYA_API_KEY` / `INAYA_APP_KEY`, **and** a user JWT
  in `TOKEN` (obtain one via guide G-01; the full example below fetches two via
  the QA sign-in shortcut — the feed is a multi-actor journey).
- `curl` plus `python3` for JSON handling (any HTTP client works; the same
  calls translate directly).

## Concepts (only what this journey needs)

- **Writes are author-bound from the JWT.** Every post/reply/sentiment/
  interaction is attributed to the JWT's `sub`; the tenant comes from the
  JWT's `tenant_id`. There is no way to write on behalf of another user over
  REST.
- **Publishing is asynchronous.** `POST /v3/feed/post` returns immediately
  with the new entry's id, but the entry is *pending*. The workflow picks it
  up (within seconds), runs the civility check, generates platform hashtags +
  sentiment, and promotes it to the published timeline. Unsafe content is
  quarantined instead. Poll the feed until your entry appears.
- **Reads only see published entries.** Both `GET /v3/feed` and
  `POST /v3/feed/search` query the published view; pending or quarantined
  entries are invisible to readers.
- **All write bodies are camelCase.** `languageCode`, `userHashtags`,
  `mediaFiles`, `parentId`, and — on sentiment/interaction — `entryId`,
  `sentimentType`, `interactionType`. Sending snake_case fails
  deserialization (`422` `missing field`).
- **Create returns a JSON string.** `POST /v3/feed/post` and `/reply` return
  the new id as a quoted JSON string (`"60OKQCQ3…"`), not an object — parse it
  accordingly.
- **Edits are author-only and wholesale for hashtags.** `PATCH` accepts new
  `content` and/or a replacement `userHashtags` list (the list you send
  *replaces* the previous one). Another user's entry → `403`; unknown entry →
  `404`.
- **Content is encrypted at rest** per-tenant; you always send and receive
  plaintext.

## Step 1 — Publish a post

One `POST` creates the entry; the quoted-string id comes back immediately:

```bash verify=auto
ALICE_TOKEN=$(curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"device_id\": \"g40-demo\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"g40-alice\"}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])')

POST_ID=$(curl -s -X POST "${FDC_URL}/v3/feed/post" \
  -H "Authorization: Bearer ${ALICE_TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"content": "Transplanted the tomato seedlings into the south beds today.", "languageCode": "en-US", "userHashtags": ["gardening", "spring"], "mediaFiles": []}' \
  | python3 -c 'import json,sys; print(json.load(sys.stdin))')
echo "post id: ${POST_ID}"
```

**Expected response** — `200` with a quoted id string (e.g.
`"60OKQCQ3A9C3EK9PB9936C2N6HBJ8HPJB0OLEI2I8K"`). `mediaFiles` is a list of
`{"filename", "dataBase64"}` objects uploaded to the CDN; leave it empty for
text-only posts.

## Step 2 — Reply, then wait for the pipeline

Replies reference a parent id. Then give the workflow a moment — it performs
three AI passes (civility, hashtags, sentiment) per pending entry:

```bash verify=auto
ALICE_TOKEN=$(curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"device_id\": \"g40-demo\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"g40-alice\"}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])')
BOB_TOKEN=$(curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"device_id\": \"g40-demo\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"g40-bob\"}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])')
# The post id from Step 1:
POST_ID=$(curl -s -X POST "${FDC_URL}/v3/feed/post" \
  -H "Authorization: Bearer ${ALICE_TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"content": "Setting up the irrigation timers for the season.", "languageCode": "en-US", "userHashtags": ["irrigation"], "mediaFiles": []}' \
  | python3 -c 'import json,sys; print(json.load(sys.stdin))')

REPLY_ID=$(curl -s -X POST "${FDC_URL}/v3/feed/reply" \
  -H "Authorization: Bearer ${BOB_TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Content-Type: application/json" \
  -d "{\"parentId\": \"${POST_ID}\", \"content\": \"Which timer model did you pick?\", \"languageCode\": \"en-US\", \"userHashtags\": [], \"mediaFiles\": []}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin))')
echo "reply id: ${REPLY_ID}"
```

**Expected response** — `200` with another quoted id. The reply validates that
the parent exists (unknown parent → `404`).

## Step 3 — Read the timeline (poll until published)

`GET /v3/feed` returns the published, ranked timeline for the caller's tenant,
paginated with `page` / `page_size`. Fresh posts take a few seconds to clear
the pipeline, so poll:

```bash verify=auto
BOB_TOKEN=$(curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"device_id\": \"g40-demo\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"g40-bob\"}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])')

for attempt in $(seq 1 24); do
  COUNT=$(curl -s "${FDQ_URL}/v3/feed?page=1&page_size=10" \
    -H "Authorization: Bearer ${BOB_TOKEN}" \
    -H "x-api-key: ${INAYA_API_KEY}" \
    -H "x-client-app-key: ${INAYA_APP_KEY}" \
    | python3 -c 'import json,sys; print(len(json.load(sys.stdin)["entries"]))')
  if [ "$COUNT" -gt 0 ]; then break; fi
  sleep 5
done
echo "entries after pipeline: ${COUNT}"

curl -s "${FDQ_URL}/v3/feed?page=1&page_size=10" \
  -H "Authorization: Bearer ${BOB_TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  | python3 -c 'import json,sys
r = json.load(sys.stdin)
for e in r["entries"][:3]:
    print("%s | %s | %s | sentiment: %s" % (e["id"][:8], e["content"][:45], e["hashtags"], e["sentiment"]))'
```

**Expected response** — once the pipeline promotes the entries (typically
10–60 s), `entries` is non-empty and each entry shows `id`, `authorId`,
`content`, `languageCode`, the combined `hashtags` (yours plus the
AI-generated platform tags), a one-word platform `sentiment`
(`Positive`/`Negative`/`Neutral`), `mediaUrls`, and `createdAt` — all
camelCase.

## Step 4 — Search the feed

`POST /v3/feed/search` filters by any combination of `author_id`, `hashtag`,
`platform_sentiment`, and full-text `query`, with the same pagination params
on the query string:

```bash verify=auto
BOB_TOKEN=$(curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"device_id\": \"g40-demo\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"g40-bob\"}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])')
AUTH=(-H "Authorization: Bearer ${BOB_TOKEN}"
      -H "x-api-key: ${INAYA_API_KEY}"
      -H "x-client-app-key: ${INAYA_APP_KEY}")

echo "-- by hashtag"
curl -s -X POST "${FDQ_URL}/v3/feed/search?page=1&page_size=10" "${AUTH[@]}" \
  -H "Content-Type: application/json" -d '{"hashtag": "irrigation"}' \
  | python3 -c 'import json,sys; print("hits:", len(json.load(sys.stdin)["entries"]))'

echo "-- full-text"
curl -s -X POST "${FDQ_URL}/v3/feed/search?page=1&page_size=10" "${AUTH[@]}" \
  -H "Content-Type: application/json" -d '{"query": "tomato seedlings"}' \
  | python3 -c 'import json,sys; print("hits:", len(json.load(sys.stdin)["entries"]))'

echo "-- sentiment filter"
curl -s -X POST "${FDQ_URL}/v3/feed/search?page=1&page_size=10" "${AUTH[@]}" \
  -H "Content-Type: application/json" -d '{"platform_sentiment": "Neutral"}' \
  | python3 -c 'import json,sys; print("hits:", len(json.load(sys.stdin)["entries"]))'
```

**Expected response** — each search returns `200` with a `{entries: [...]}`
body; hashtag matches on user or platform tags, `query` is PostgreSQL
full-text (`plainto_tsquery`), and `platform_sentiment` matches the workflow's
AI-assigned label. A search that matches nothing returns `entries: []`, not an
error.

## Step 5 — Edit (author only) and react

`PATCH` is author-only; other users get `403`, unknown entries `404`. The
sentiment and interaction bodies are camelCase too (`entryId`, `sentimentType`,
`interactionType`):

```bash verify=auto
ALICE_TOKEN=$(curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"device_id\": \"g40-demo\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"g40-alice\"}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])')
BOB_TOKEN=$(curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"device_id\": \"g40-demo\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"g40-bob\"}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])')
AUTH=(-H "x-api-key: ${INAYA_API_KEY}" -H "x-client-app-key: ${INAYA_APP_KEY}")

# A fresh post owned by alice:
POST_ID=$(curl -s -X POST "${FDC_URL}/v3/feed/post" \
  -H "Authorization: Bearer ${ALICE_TOKEN}" "${AUTH[@]}" \
  -H "Content-Type: application/json" \
  -d '{"content": "Testing the solar fence charger.", "languageCode": "en-US", "userHashtags": ["fencing"], "mediaFiles": []}' \
  | python3 -c 'import json,sys; print(json.load(sys.stdin))')

echo "-- bob (non-author) patches -> HTTP $(curl -s -o /dev/null -w '%{http_code}' \
  -X PATCH "${FDC_URL}/v3/feed/post/${POST_ID}" \
  -H "Authorization: Bearer ${BOB_TOKEN}" "${AUTH[@]}" \
  -H "Content-Type: application/json" \
  -d '{"content": "hacked"}')"
echo "-- alice (author) patches -> $(curl -s -X PATCH "${FDC_URL}/v3/feed/post/${POST_ID}" \
  -H "Authorization: Bearer ${ALICE_TOKEN}" "${AUTH[@]}" \
  -H "Content-Type: application/json" \
  -d '{"content": "Testing the solar fence charger — holding at 7.2 kV.", "userHashtags": ["fencing", "solar"]}')"
echo "-- unknown entry -> HTTP $(curl -s -o /dev/null -w '%{http_code}' \
  -X PATCH "${FDC_URL}/v3/feed/post/no-such-entry" \
  -H "Authorization: Bearer ${ALICE_TOKEN}" "${AUTH[@]}" \
  -H "Content-Type: application/json" \
  -d '{"content": "x"}')"
REACT_CODE=$(curl -s -o /tmp/g40-react.json -w '%{http_code}' \
  -X POST "${FDC_URL}/v3/feed/sentiment" \
  -H "Authorization: Bearer ${BOB_TOKEN}" "${AUTH[@]}" \
  -H "Content-Type: application/json" \
  -d "{\"entryId\": \"${POST_ID}\", \"sentimentType\": \"Like\"}")
echo "-- bob reacts -> HTTP ${REACT_CODE} $(cat /tmp/g40-react.json)"
test "$(cat /tmp/g40-react.json)" = '"Success"' || { echo "sentiment failed"; exit 1; }
VIEW_CODE=$(curl -s -o /tmp/g40-view.json -w '%{http_code}' \
  -X POST "${FDC_URL}/v3/feed/interaction" \
  -H "Authorization: Bearer ${BOB_TOKEN}" "${AUTH[@]}" \
  -H "Content-Type: application/json" \
  -d "{\"entryId\": \"${POST_ID}\", \"interactionType\": \"View\"}")
echo "-- bob views -> HTTP ${VIEW_CODE} $(cat /tmp/g40-view.json)"
test "$(cat /tmp/g40-view.json)" = '"Success"' || { echo "interaction failed"; exit 1; }
```

**Expected response** — non-author patch → `403`; author patch → `200`
`"Success"` (content and hashtags updated; the platform tags are untouched);
unknown entry → `404`; sentiment and interaction both → `200` `"Success"`.

## Step 6 — Delete (author only, permanent)

`DELETE` removes the entry with its content, sentiments, and interaction
history. Only the author may delete:

```bash verify=auto
ALICE_TOKEN=$(curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"device_id\": \"g40-demo\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"g40-alice\"}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])')
AUTH=(-H "Authorization: Bearer ${ALICE_TOKEN}"
      -H "x-api-key: ${INAYA_API_KEY}"
      -H "x-client-app-key: ${INAYA_APP_KEY}")

POST_ID=$(curl -s -X POST "${FDC_URL}/v3/feed/post" "${AUTH[@]}" \
  -H "Content-Type: application/json" \
  -d '{"content": "Temporary entry to be deleted.", "languageCode": "en-US", "userHashtags": [], "mediaFiles": []}' \
  | python3 -c 'import json,sys; print(json.load(sys.stdin))')

curl -s -o /dev/null -w "delete -> HTTP %{http_code}\n" \
  -X DELETE "${FDC_URL}/v3/feed/post/${POST_ID}" "${AUTH[@]}"
curl -s -o /dev/null -w "delete again -> HTTP %{http_code}\n" \
  -X DELETE "${FDC_URL}/v3/feed/post/${POST_ID}" "${AUTH[@]}"
```

**Expected response** — `delete -> HTTP 204`; a second delete returns `404`
(the entry is gone, so the author check has nothing to authorize).

## Full working example

A complete publish → poll → search → react → delete cycle
(`feed_publish.sh`) — self-contained via the QA shortcut (local/QA environments
only; production uses the G-01 flows):

```bash verify=auto
#!/usr/bin/env bash
set -euo pipefail

KEYS=(-H "x-api-key: ${INAYA_API_KEY}" -H "x-client-app-key: ${INAYA_APP_KEY}")
login() {
  curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" "${KEYS[@]}" \
    -H "Content-Type: application/json" \
    -d "{\"device_id\": \"g40-demo\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"$1\"}" \
    | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])'
}
ALICE=$(login g40-alice); BOB=$(login g40-bob)

# 1. Alice publishes, Bob replies
POST_ID=$(curl -s -X POST "${FDC_URL}/v3/feed/post" \
  -H "Authorization: Bearer ${ALICE}" "${KEYS[@]}" \
  -H "Content-Type: application/json" \
  -d '{"content": "First cutting of hay is baled and barned.", "languageCode": "en-US", "userHashtags": ["hay-season"], "mediaFiles": []}' \
  | python3 -c 'import json,sys; print(json.load(sys.stdin))')
echo "published: ${POST_ID}"
curl -sf -o /dev/null -X POST "${FDC_URL}/v3/feed/reply" \
  -H "Authorization: Bearer ${BOB}" "${KEYS[@]}" \
  -H "Content-Type: application/json" \
  -d "{\"parentId\": \"${POST_ID}\", \"content\": \"How many bales?\", \"languageCode\": \"en-US\", \"userHashtags\": [], \"mediaFiles\": []}"
echo "replied"

# 2. Poll the timeline until the pipeline publishes
for attempt in $(seq 1 24); do
  if curl -sf "${FDQ_URL}/v3/feed?page=1&page_size=10" \
      -H "Authorization: Bearer ${BOB}" "${KEYS[@]}" \
      | grep -q "hay is baled"; then
    echo "visible on the feed"; break
  fi
  sleep 5
done

# 3. Search and react
curl -sf -X POST "${FDQ_URL}/v3/feed/search?page=1&page_size=10" \
  -H "Authorization: Bearer ${BOB}" "${KEYS[@]}" \
  -H "Content-Type: application/json" \
  -d '{"hashtag": "hay-season"}' | grep -q "baled" && echo "search found it"
curl -sf -o /dev/null -X POST "${FDC_URL}/v3/feed/sentiment" \
  -H "Authorization: Bearer ${BOB}" "${KEYS[@]}" \
  -H "Content-Type: application/json" \
  -d "{\"entryId\": \"${POST_ID}\", \"sentimentType\": \"Like\"}" && echo "reacted"

# 4. Alice deletes her post
curl -sf -o /dev/null -X DELETE "${FDC_URL}/v3/feed/post/${POST_ID}" \
  -H "Authorization: Bearer ${ALICE}" "${KEYS[@]}" && echo "deleted"
```

(`AUTH_URL` is the Auth Service base URL — `http://127.0.0.1:40004` on the
local stack; `FDC_URL` / `FDQ_URL` are the Feed Command / Feed Query base URLs
— `http://127.0.0.1:40167` / `http://127.0.0.1:40168` locally; `TENANT_ID` is
your tenant's id.)

## Try it live

Open the [Playground](../../playgrounds/playground.html#communication) →
**Communication** — the *Feed* console exercises the same publish → timeline
→ search cycle against the live Feed services.

## Copy agent prompt

```prompt
You are working with the Inaya Labs Feed platform. Feed Command (writes) is at
FDC_URL and Feed Query (reads) at FDQ_URL; the Auth Service is at AUTH_URL;
the tenant key pair is in INAYA_API_KEY / INAYA_APP_KEY and TENANT_ID. Obtain
two user JWTs via POST /v3/auth/qa/login {"device_id","locale","tenant_id",
"username"} with usernames "g40-agent-a" and "g40-agent-b" if TOKENs are not
already set. Steps: (1) As agent-a, POST FDC /v3/feed/post {"content",
"languageCode": "en-US", "userHashtags": ["agent-demo"], "mediaFiles": []}
(the response is a quoted JSON string — that is the post id) and report it.
(2) As agent-b, POST FDC /v3/feed/reply with that parentId and a short
question. (3) Poll FDQ GET /v3/feed?page=1&page_size=10 as agent-b every 5
seconds until the post appears (the workflow publishes asynchronously), then
report the entry's hashtags and platform sentiment. (4) Search FDQ POST
/v3/feed/search with {"hashtag": "agent-demo"} and report the hit count.
(5) As agent-b, attempt to PATCH FDC /v3/feed/post/{id} and report the 403;
then as agent-a PATCH new content and report the 200. (6) As agent-b, POST
FDC /v3/feed/sentiment {"entryId": "<id>", "sentimentType": "Like"} — note
the camelCase fields — and report the 200. (7) As agent-a, DELETE
/v3/feed/post/{id} and report the 204, then repeat and report the 404. Never
log the JWTs; report status codes per step.
```

## API reference

- [Feed Command — Swagger UI](https://feed-command-service-sandbox.inayalabs.com/v3/feed/swagger-ui/) · [Feed Query — Swagger UI](https://feed-query-service-sandbox.inayalabs.com/v3/feed/swagger-ui/)

## Next journeys

- **Direct messaging between users** (G-41) — private conversations on the Messaging CQRS pair.
- **Managing users & profiles** (G-30) — the user records every author maps back to.
- **Authentication & sessions** (G-01) — the token lifecycle that every call in this guide depends on.

### G-41 · Direct messaging between users
URL: /docs/guides/direct-messaging-users.html

> Create a conversation between users, send messages that pass through the moderation pipeline, read and poll the thread, edit and react, track delivery/read receipts, and clean up — across the Messaging Command / Messaging Query CQRS pair.

> **Journey.** As a developer, I want my app's users to open conversations with
> each other, exchange messages, see delivery/read state, and edit or remove
> what they sent — while the platform moderates content and translates
> messages for each reader automatically.

The Messaging platform is a CQRS trio: **Messaging Command** (MSC) owns every
write, **Messaging Query** (MSQ) owns every read, and **Messaging Workflow**
(MSW) runs the pipeline in the background — a civility check via Evergreen AI,
sentiment analysis, then promotion to the published view (with an SSE
`MessagePublished` fan-out via the Notifications service so clients can update
in realtime). This guide walks create conversation → send → read/poll → edit →
react/receipts → delete, verified against the local stack (MSC `:40163`, MSQ
`:40164`); the shared dev subdomains behave identically.

## Prerequisites

- Messaging Command, Messaging Query **and** Messaging Workflow are reachable
  (local stack `:40163`/`:40164` + workflow running, or the shared dev
  subdomains). The workflow is what publishes messages — without it, writes
  succeed but recipients never see them.
- The Notifications service must be reachable by the workflow (it carries the
  internal publish + SSE subscription); Evergreen AI must be able to serve the
  moderation model (on macOS it resolves via Ollama).
- Your tenant key pair in `INAYA_API_KEY` / `INAYA_APP_KEY`, **and** user JWTs
  (obtain via guide G-01; the snippets below fetch fresh users via the QA
  sign-in shortcut — messaging is inherently multi-actor).
- `curl` plus `python3` for JSON handling (any HTTP client works; the same
  calls translate directly).

## Concepts (only what this journey needs)

- **Everything is camelCase** — request bodies (`conversationId`, `otherParticipants`,
  `messageId`, `mediaUrls`) and response fields (`senderId`, `publishedAt`).
  The one exception is the pagination query string: `page` and `page_size`
  (snake_case — `?page=1&page_size=20`, defaults 1 / 20, max 100).
- **The sender comes from the JWT.** There is no "on behalf of" parameter:
  the sender is the JWT's `sub`, the tenant is the JWT's `tenant_id`.
  Participants are referenced by **user id** (the Users service `id`, equal to
  the JWT `sub`) — never by username.
- **Sending is async.** `POST /v3/messaging/messages` returns `201`
  immediately with `content: null` and `publishedAt: null` — the message is
  pending moderation. The workflow promotes it seconds later; only then do
  *other* participants see it. The **author always sees their own** pending,
  published, *and* quarantined messages; everyone else sees published only.
- **Reactions and receipts target published messages.** A sentiment or receipt
  posted against a still-pending message is rejected as `404` — clients should
  react only to messages the user can actually see.
- **Reads are per-reader localized.** `content`/`contentLocale`/`isOriginal`
  in responses are the *caller's* localized version: exact locale → language
  fallback → AI translation via Linga (cached). `isOriginal: true` means the
  message was written in your locale.
- **Errors are often bare status codes.** Several endpoints answer `401` /
  `403` / `404` with an empty body rather than a JSON error document — check
  the status code first.
- **Content is encrypted at rest** per-tenant (AES-256); you always send and
  receive plaintext.

### Base URLs used below

| Service | Local | Variable |
|---|---|---|
| Auth (QA sign-in) | `http://127.0.0.1:40004` | `AUTH_URL` |
| Messaging Command (writes) | `http://127.0.0.1:40163` | `MSC_URL` |
| Messaging Query (reads) | `http://127.0.0.1:40164` | `MSQ_URL` |

All messaging endpoints are prefixed `/v3/messaging` and require the tenant
key pair headers plus `Authorization: Bearer <JWT>`.

## Step 1 — Create a conversation

A conversation is a real entity with a participant list (roles `Owner` /
`ConversationManager` / `Member`, max 100). The creator becomes `Owner`;
`otherParticipants` takes the **user ids** of everyone else. Creating again
with the same title + participant set is idempotent — you get the existing
conversation back:

```bash verify=auto
login() {  # username -> JWT
  curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
    -H 'Content-Type: application/json' \
    -H "x-api-key: ${INAYA_API_KEY}" \
    -H "x-client-app-key: ${INAYA_APP_KEY}" \
    -d "{\"device_id\": \"g41-demo-device\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"$1\"}" \
    | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])'
}
user_id() {  # username -> user id
  curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
    -H 'Content-Type: application/json' \
    -H "x-api-key: ${INAYA_API_KEY}" \
    -H "x-client-app-key: ${INAYA_APP_KEY}" \
    -d "{\"device_id\": \"g41-demo-device\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"$1\"}" \
    | python3 -c 'import json,sys; print(json.load(sys.stdin)["user"]["id"])'
}
KEYS=(-H "x-api-key: ${INAYA_API_KEY}" -H "x-client-app-key: ${INAYA_APP_KEY}")
ALICE_T=$(login g41-alice); BOB_T=$(login g41-bob); CAROL_T=$(login g41-carol)
ALICE_U=$(user_id g41-alice); BOB_U=$(user_id g41-bob)

BODY=$(printf '{"otherParticipants": ["%s"], "title": "G-41 demo"}' "$BOB_U")
RESP=$(curl -s -w '\n%{http_code}' -X POST "${MSC_URL}/v3/messaging/conversations" \
  -H "Authorization: Bearer ${ALICE_T}" "${KEYS[@]}" \
  -H 'Content-Type: application/json' -d "$BODY")
CODE=$(tail -n1 <<< "$RESP"); echo "create -> HTTP ${CODE}"
test "$CODE" = 201 || { echo "expected 201"; exit 1; }
CONV=$(sed '$d' <<< "$RESP")
echo "$CONV" | python3 -c 'import json,sys; d=json.load(sys.stdin); print("conversation id:", d["id"]); print("createdById:", d["createdById"])'
CID=$(echo "$CONV" | python3 -c 'import json,sys; print(json.load(sys.stdin)["id"])')

# Idempotent: same title + participants returns the same conversation:
CID2=$(curl -s -X POST "${MSC_URL}/v3/messaging/conversations" \
  -H "Authorization: Bearer ${ALICE_T}" "${KEYS[@]}" \
  -H 'Content-Type: application/json' -d "$BODY" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["id"])')
echo "re-create returns same id: $([ "$CID" = "$CID2" ] && echo yes || echo NO)"
test "$CID" = "$CID2" || exit 1

# Non-participant carol cannot open the conversation:
CODE=$(curl -s -o /dev/null -w '%{http_code}' "${MSQ_URL}/v3/messaging/conversations/${CID}" \
  -H "Authorization: Bearer ${CAROL_T}" "${KEYS[@]}")
echo "carol (non-participant) reads -> HTTP ${CODE}"
test "$CODE" -ge 400 || { echo "expected 4xx"; exit 1; }
```

**Expected response** — create → `201` with the `Conversation` JSON (`id`,
`title`, `createdById` = alice, `latestMessageId: null`); re-create → the same
`id`; carol → `4xx` with an empty body.

## Step 2 — Send a message

The sender is whoever the JWT belongs to. The `201` response is the stored
message **before** moderation — note `content: null` and `publishedAt: null`:

```bash verify=auto
login() {
  curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
    -H 'Content-Type: application/json' \
    -H "x-api-key: ${INAYA_API_KEY}" \
    -H "x-client-app-key: ${INAYA_APP_KEY}" \
    -d "{\"device_id\": \"g41-demo-device\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"$1\"}" \
    | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])'
}
user_id() {
  curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
    -H 'Content-Type: application/json' \
    -H "x-api-key: ${INAYA_API_KEY}" \
    -H "x-client-app-key: ${INAYA_APP_KEY}" \
    -d "{\"device_id\": \"g41-demo-device\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"$1\"}" \
    | python3 -c 'import json,sys; print(json.load(sys.stdin)["user"]["id"])'
}
KEYS=(-H "x-api-key: ${INAYA_API_KEY}" -H "x-client-app-key: ${INAYA_APP_KEY}")
ALICE_T=$(login g41-alice); CAROL_T=$(login g41-carol)
BOB_U=$(user_id g41-bob)
ALICE_U=$(user_id g41-alice)

# Fresh conversation for this step:
BODY=$(printf '{"otherParticipants": ["%s"], "title": "G-41 step2"}' "$BOB_U")
CID=$(curl -s -X POST "${MSC_URL}/v3/messaging/conversations" \
  -H "Authorization: Bearer ${ALICE_T}" "${KEYS[@]}" \
  -H 'Content-Type: application/json' -d "$BODY" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["id"])')

MSG_BODY=$(printf '{"conversationId": "%s", "content": "Morning! Watering the tomatoes before it gets hot.", "mediaUrls": []}' "$CID")
MSG=$(curl -s -w '\n%{http_code}' -X POST "${MSC_URL}/v3/messaging/messages" \
  -H "Authorization: Bearer ${ALICE_T}" "${KEYS[@]}" \
  -H 'Content-Type: application/json' -d "$MSG_BODY")
CODE=$(tail -n1 <<< "$MSG"); echo "send -> HTTP ${CODE}"
test "$CODE" = 201 || { echo "expected 201"; exit 1; }
echo "$MSG" | sed '$d' | python3 -c '
import json,sys
d = json.load(sys.stdin)
print("id:", d["id"])
print("senderId:", d["senderId"])
print("content (pending):", d["content"], "| publishedAt:", d["publishedAt"])
assert d["content"] is None and d["publishedAt"] is None, "new message must be unpublished"'

# Non-participant carol cannot send into the conversation:
CAROL_BODY=$(printf '{"conversationId": "%s", "content": "intrusion", "mediaUrls": []}' "$CID")
CODE=$(curl -s -o /dev/null -w '%{http_code}' -X POST "${MSC_URL}/v3/messaging/messages" \
  -H "Authorization: Bearer ${CAROL_T}" "${KEYS[@]}" \
  -H 'Content-Type: application/json' -d "$CAROL_BODY")
echo "carol (non-participant) sends -> HTTP ${CODE}"
test "$CODE" = 401 || { echo "expected 401"; exit 1; }
```

**Expected response** — alice → `201`, the message JSON with `senderId` = her
user id, `content: null`, `publishedAt: null` (it appears once the workflow
publishes it); carol → bare `401`.

## Step 3 — Read the thread, poll for new messages

Recipients read via Messaging Query. Messages come back **newest first**, with
the caller's localized `content` filled in once published. Poll the list until
your message appears, then use `since` to fetch only newer messages on
subsequent polls:

```bash verify=auto
login() {
  curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
    -H 'Content-Type: application/json' \
    -H "x-api-key: ${INAYA_API_KEY}" \
    -H "x-client-app-key: ${INAYA_APP_KEY}" \
    -d "{\"device_id\": \"g41-demo-device\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"$1\"}" \
    | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])'
}
user_id() {
  curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
    -H 'Content-Type: application/json' \
    -H "x-api-key: ${INAYA_API_KEY}" \
    -H "x-client-app-key: ${INAYA_APP_KEY}" \
    -d "{\"device_id\": \"g41-demo-device\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"$1\"}" \
    | python3 -c 'import json,sys; print(json.load(sys.stdin)["user"]["id"])'
}
KEYS=(-H "x-api-key: ${INAYA_API_KEY}" -H "x-client-app-key: ${INAYA_APP_KEY}")
ALICE_T=$(login g41-alice); BOB_T=$(login g41-bob)
BOB_U=$(user_id g41-bob); ALICE_U=$(user_id g41-alice)

BODY=$(printf '{"otherParticipants": ["%s"], "title": "G-41 step3"}' "$BOB_U")
CID=$(curl -s -X POST "${MSC_URL}/v3/messaging/conversations" \
  -H "Authorization: Bearer ${ALICE_T}" "${KEYS[@]}" \
  -H 'Content-Type: application/json' -d "$BODY" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["id"])')
MSG1_BODY=$(printf '{"conversationId": "%s", "content": "First: moved the cows to the north pasture.", "mediaUrls": []}' "$CID")
MID1=$(curl -s -X POST "${MSC_URL}/v3/messaging/messages" \
  -H "Authorization: Bearer ${ALICE_T}" "${KEYS[@]}" \
  -H 'Content-Type: application/json' -d "$MSG1_BODY" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["id"])')

# Poll as bob until the workflow publishes it (seconds, not minutes):
FOUND=""
for i in $(seq 1 24); do
  FOUND=$(curl -s "${MSQ_URL}/v3/messaging/conversations/${CID}/messages?page=1&page_size=10" \
    -H "Authorization: Bearer ${BOB_T}" "${KEYS[@]}" \
    | python3 -c 'import json,sys
ms = json.load(sys.stdin)
m = next((m for m in ms if m["id"] == "'"$MID1"'"), None)
print(m["content"] or "" if m else "")')
  [ -n "$FOUND" ] && break
  sleep 5
done
echo "bob sees: $FOUND"
grep -q "north pasture" <<< "$FOUND" || { echo "message never published"; exit 1; }

# A second message; bob's client then polls with `since` (ISO 8601).
# `since` filters on created_at — capture the watermark BEFORE sending:
SINCE=$(date -u +%Y-%m-%dT%H:%M:%SZ)
sleep 1
MSG2_BODY=$(printf '{"conversationId": "%s", "content": "Second: fence charger reads 7.2 kV.", "mediaUrls": []}' "$CID")
MID2=$(curl -s -X POST "${MSC_URL}/v3/messaging/messages" \
  -H "Authorization: Bearer ${ALICE_T}" "${KEYS[@]}" \
  -H 'Content-Type: application/json' -d "$MSG2_BODY" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["id"])')
# Poll the incremental view until the new message is published:
for i in $(seq 1 24); do
  NEWS=$(curl -s "${MSQ_URL}/v3/messaging/conversations/${CID}/messages?page=1&page_size=10&since=${SINCE}" \
    -H "Authorization: Bearer ${BOB_T}" "${KEYS[@]}")
  if echo "$NEWS" | grep -q "fence charger"; then break; fi
  sleep 5
done
echo "since ${SINCE} -> $(echo "$NEWS" | python3 -c 'import json,sys
ms = json.load(sys.stdin)
print(len(ms), "message(s):", [ (m["content"] or "")[:30] for m in ms ])')"
echo "$NEWS" | python3 -c 'import json,sys
ms = json.load(sys.stdin)
assert any("fence charger" in (m["content"] or "") for m in ms), "since-poll should include the new message"
assert all("north pasture" not in (m["content"] or "") for m in ms), "since-poll should exclude the older message"'
```

**Expected response** — bob's poll eventually returns the message with its
plaintext `content` and a `publishedAt` timestamp; the `since` poll returns
only the newer message. Newest-first ordering: when both messages are present,
the fence-charger message (created later) comes before the pasture message.

## Step 4 — Edit a message (author only)

`PUT /v3/messaging/messages` replaces the content. Non-authors get `403`;
unknown messages `404`. The edit re-enters the moderation pipeline — the
recipient sees the updated content only after the workflow re-publishes:

```bash verify=auto
login() {
  curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
    -H 'Content-Type: application/json' \
    -H "x-api-key: ${INAYA_API_KEY}" \
    -H "x-client-app-key: ${INAYA_APP_KEY}" \
    -d "{\"device_id\": \"g41-demo-device\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"$1\"}" \
    | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])'
}
user_id() {
  curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
    -H 'Content-Type: application/json' \
    -H "x-api-key: ${INAYA_API_KEY}" \
    -H "x-client-app-key: ${INAYA_APP_KEY}" \
    -d "{\"device_id\": \"g41-demo-device\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"$1\"}" \
    | python3 -c 'import json,sys; print(json.load(sys.stdin)["user"]["id"])'
}
KEYS=(-H "x-api-key: ${INAYA_API_KEY}" -H "x-client-app-key: ${INAYA_APP_KEY}")
ALICE_T=$(login g41-alice); BOB_T=$(login g41-bob)
BOB_U=$(user_id g41-bob)

BODY=$(printf '{"otherParticipants": ["%s"], "title": "G-41 step4"}' "$BOB_U")
CID=$(curl -s -X POST "${MSC_URL}/v3/messaging/conversations" \
  -H "Authorization: Bearer ${ALICE_T}" "${KEYS[@]}" \
  -H 'Content-Type: application/json' -d "$BODY" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["id"])')
M_BODY=$(printf '{"conversationId": "%s", "content": "Eggs collected, 14 today.", "mediaUrls": []}' "$CID")
MID=$(curl -s -X POST "${MSC_URL}/v3/messaging/messages" \
  -H "Authorization: Bearer ${ALICE_T}" "${KEYS[@]}" \
  -H 'Content-Type: application/json' -d "$M_BODY" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["id"])')

# Bob (non-author) tries to edit -> 403:
EDIT_BODY=$(printf '{"messageId": "%s", "content": "tampered"}' "$MID")
CODE=$(curl -s -o /dev/null -w '%{http_code}' -X PUT "${MSC_URL}/v3/messaging/messages" \
  -H "Authorization: Bearer ${BOB_T}" "${KEYS[@]}" \
  -H 'Content-Type: application/json' -d "$EDIT_BODY")
echo "bob edits alice's message -> HTTP ${CODE}"
test "$CODE" = 403 || { echo "expected 403"; exit 1; }

# Unknown message -> 404:
CODE=$(curl -s -o /dev/null -w '%{http_code}' -X PUT "${MSC_URL}/v3/messaging/messages" \
  -H "Authorization: Bearer ${ALICE_T}" "${KEYS[@]}" \
  -H 'Content-Type: application/json' \
  -d '{"messageId": "no-such-message", "content": "x"}')
echo "alice edits unknown message -> HTTP ${CODE}"
test "$CODE" = 404 || { echo "expected 404"; exit 1; }

# Alice edits her own -> 200. The response is transitional (content null
# while the workflow re-runs translation/moderation); assert id + status:
EDIT_BODY=$(printf '{"messageId": "%s", "content": "Eggs collected, 16 today — found a second nest."}' "$MID")
EDITED=$(curl -s -w '\n%{http_code}' -X PUT "${MSC_URL}/v3/messaging/messages" \
  -H "Authorization: Bearer ${ALICE_T}" "${KEYS[@]}" \
  -H 'Content-Type: application/json' -d "$EDIT_BODY")
CODE=$(tail -n1 <<< "$EDITED"); echo "alice edits her own -> HTTP ${CODE}"
test "$CODE" = 200 || { echo "expected 200"; exit 1; }
echo "$EDITED" | sed '$d' | python3 -c 'import json,sys
d = json.load(sys.stdin)
assert d["id"] == "'"$MID"'", "response carries the edited message id"
print("edit accepted for message", d["id"], "| content (transitional):", d["content"])'

# Recipient sees it after the workflow re-publishes the edit:
SEEN=""
for i in $(seq 1 24); do
  SEEN=$(curl -s "${MSQ_URL}/v3/messaging/conversations/${CID}/messages?page=1&page_size=10" \
    -H "Authorization: Bearer ${BOB_T}" "${KEYS[@]}" \
    | python3 -c 'import json,sys
ms = json.load(sys.stdin)
m = next((m for m in ms if m["id"] == "'"$MID"'"), None)
print((m["content"] or "") if m else "")')
  grep -q "second nest" <<< "$SEEN" && break
  sleep 5
done
echo "bob eventually sees: $SEEN"
grep -q "second nest" <<< "$SEEN" || { echo "edit never published"; exit 1; }
```

**Expected response** — bob → `403`; unknown id → `404`; alice → `200` carrying
the edited message id (the response content is transitional — `null` until the
workflow re-runs translation); bob's poll converges on the new content.

## Step 5 — React with sentiment and track receipts

Users attach a sentiment (`Positive`/`Negative`/`Neutral`/`Mixed` + intensity
`0.0`–`1.0`) to any message they can see, and post delivery/read receipts for
themselves. Receipts are per-user state you read back per message:

```bash verify=auto
login() {
  curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
    -H 'Content-Type: application/json' \
    -H "x-api-key: ${INAYA_API_KEY}" \
    -H "x-client-app-key: ${INAYA_APP_KEY}" \
    -d "{\"device_id\": \"g41-demo-device\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"$1\"}" \
    | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])'
}
user_id() {
  curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
    -H 'Content-Type: application/json' \
    -H "x-api-key: ${INAYA_API_KEY}" \
    -H "x-client-app-key: ${INAYA_APP_KEY}" \
    -d "{\"device_id\": \"g41-demo-device\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"$1\"}" \
    | python3 -c 'import json,sys; print(json.load(sys.stdin)["user"]["id"])'
}
KEYS=(-H "x-api-key: ${INAYA_API_KEY}" -H "x-client-app-key: ${INAYA_APP_KEY}")
ALICE_T=$(login g41-alice); BOB_T=$(login g41-bob)
BOB_U=$(user_id g41-bob)

BODY=$(printf '{"otherParticipants": ["%s"], "title": "G-41 step5"}' "$BOB_U")
CID=$(curl -s -X POST "${MSC_URL}/v3/messaging/conversations" \
  -H "Authorization: Bearer ${ALICE_T}" "${KEYS[@]}" \
  -H 'Content-Type: application/json' -d "$BODY" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["id"])')
M_BODY=$(printf '{"conversationId": "%s", "content": "The well pump is holding steady at 42 psi.", "mediaUrls": []}' "$CID")
MID=$(curl -s -X POST "${MSC_URL}/v3/messaging/messages" \
  -H "Authorization: Bearer ${ALICE_T}" "${KEYS[@]}" \
  -H 'Content-Type: application/json' -d "$M_BODY" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["id"])')

# Reactions target *visible* messages — wait until the workflow publishes
# THIS message (match by id: the idempotent conversation may hold older ones):
for i in $(seq 1 24); do
  VISIBLE=$(curl -s "${MSQ_URL}/v3/messaging/conversations/${CID}/messages?page=1&page_size=10" \
    -H "Authorization: Bearer ${BOB_T}" "${KEYS[@]}" \
    | python3 -c 'import json,sys
ms = json.load(sys.stdin)
print("yes" if any(m["id"] == "'"$MID"'" for m in ms) else "")')
  [ "$VISIBLE" = yes ] && { echo "published"; break; }
  sleep 5
done
test "$VISIBLE" = yes || { echo "message never published"; exit 1; }

# Bob reacts with a sentiment:
SENT_BODY=$(printf '{"messageId": "%s", "sentiment": "Positive", "intensity": 0.8}' "$MID")
CODE=$(curl -s -o /dev/null -w '%{http_code}' -X POST "${MSC_URL}/v3/messaging/messages/sentiment" \
  -H "Authorization: Bearer ${BOB_T}" "${KEYS[@]}" \
  -H 'Content-Type: application/json' -d "$SENT_BODY")
echo "bob reacts -> HTTP ${CODE}"
test "$CODE" = 200 || { echo "expected 200"; exit 1; }

# Read it back (per user):
curl -s "${MSQ_URL}/v3/messaging/messages/${MID}/sentiment/${BOB_U}" \
  -H "Authorization: Bearer ${ALICE_T}" "${KEYS[@]}" \
  | python3 -c 'import json,sys
d = json.load(sys.stdin)
print("sentiment:", d["sentiment"], "| intensity:", d["intensity"], "| userId:", d["userId"])
assert d["sentiment"] == "Positive" and d["userId"]'

# Bob posts a delivery receipt, then a read receipt:
CODE=$(curl -s -o /dev/null -w '%{http_code}' -X POST "${MSC_URL}/v3/messaging/messages/${MID}/delivery-receipts" \
  -H "Authorization: Bearer ${BOB_T}" "${KEYS[@]}")
echo "delivery receipt -> HTTP ${CODE}"
test "$CODE" = 200 || { echo "expected 200"; exit 1; }
CODE=$(curl -s -o /dev/null -w '%{http_code}' -X POST "${MSC_URL}/v3/messaging/messages/${MID}/read-receipts" \
  -H "Authorization: Bearer ${BOB_T}" "${KEYS[@]}")
echo "read receipt -> HTTP ${CODE}"
test "$CODE" = 200 || { echo "expected 200"; exit 1; }

# Read the receipt state back:
curl -s "${MSQ_URL}/v3/messaging/messages/${MID}/receipts?page=1&page_size=10" \
  -H "Authorization: Bearer ${ALICE_T}" "${KEYS[@]}" \
  | python3 -c 'import json,sys
rs = json.load(sys.stdin)
for r in rs:
    print("receipt:", r["userId"][:12], "->", r["status"])
assert any(r["status"] == "DeliveredAndRead" for r in rs), "bob read state should be visible"'
```

**Expected response** — sentiment and both receipts → `200` (empty body); the
sentiment read-back returns the `UserMessageSentiment` JSON; the receipts list
shows bob's message as `DeliveredAndRead` (the final status supersedes
`Delivered`).

## Step 6 — Delete a message (author only, permanent)

`DELETE` removes the message for everyone. Author (or an admin role) only,
`204` on success, `404` once already gone:

```bash verify=auto
login() {
  curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
    -H 'Content-Type: application/json' \
    -H "x-api-key: ${INAYA_API_KEY}" \
    -H "x-client-app-key: ${INAYA_APP_KEY}" \
    -d "{\"device_id\": \"g41-demo-device\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"$1\"}" \
    | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])'
}
user_id() {
  curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
    -H 'Content-Type: application/json' \
    -H "x-api-key: ${INAYA_API_KEY}" \
    -H "x-client-app-key: ${INAYA_APP_KEY}" \
    -d "{\"device_id\": \"g41-demo-device\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"$1\"}" \
    | python3 -c 'import json,sys; print(json.load(sys.stdin)["user"]["id"])'
}
KEYS=(-H "x-api-key: ${INAYA_API_KEY}" -H "x-client-app-key: ${INAYA_APP_KEY}")
ALICE_T=$(login g41-alice); BOB_T=$(login g41-bob)
BOB_U=$(user_id g41-bob)

BODY=$(printf '{"otherParticipants": ["%s"], "title": "G-41 step6"}' "$BOB_U")
CID=$(curl -s -X POST "${MSC_URL}/v3/messaging/conversations" \
  -H "Authorization: Bearer ${ALICE_T}" "${KEYS[@]}" \
  -H 'Content-Type: application/json' -d "$BODY" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["id"])')
M_BODY=$(printf '{"conversationId": "%s", "content": "Scratch note, deleting in a moment.", "mediaUrls": []}' "$CID")
MID=$(curl -s -X POST "${MSC_URL}/v3/messaging/messages" \
  -H "Authorization: Bearer ${ALICE_T}" "${KEYS[@]}" \
  -H 'Content-Type: application/json' -d "$M_BODY" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["id"])')

CODE=$(curl -s -o /dev/null -w '%{http_code}' -X DELETE "${MSC_URL}/v3/messaging/messages/${MID}" \
  -H "Authorization: Bearer ${BOB_T}" "${KEYS[@]}")
echo "bob (non-author) deletes -> HTTP ${CODE}"
test "$CODE" = 403 || { echo "expected 403"; exit 1; }

CODE=$(curl -s -o /dev/null -w '%{http_code}' -X DELETE "${MSC_URL}/v3/messaging/messages/${MID}" \
  -H "Authorization: Bearer ${ALICE_T}" "${KEYS[@]}")
echo "alice (author) deletes -> HTTP ${CODE}"
test "$CODE" = 204 || { echo "expected 204"; exit 1; }

CODE=$(curl -s -o /dev/null -w '%{http_code}' -X DELETE "${MSC_URL}/v3/messaging/messages/${MID}" \
  -H "Authorization: Bearer ${ALICE_T}" "${KEYS[@]}")
echo "delete again -> HTTP ${CODE}"
test "$CODE" -ge 400 || { echo "expected 4xx on repeat"; exit 1; }
```

**Expected response** — bob → `403`; alice → `204`; repeat → `404` (bare code,
empty body).

## Full example — one condensed pass

```bash verify=auto
set -e
KEYS=(-H "x-api-key: ${INAYA_API_KEY}" -H "x-client-app-key: ${INAYA_APP_KEY}")
jwt() {
  curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
    -H 'Content-Type: application/json' "${KEYS[@]}" \
    -d "{\"device_id\": \"g41-full-device\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"$1\"}" \
    | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])'
}
uid() {
  curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
    -H 'Content-Type: application/json' "${KEYS[@]}" \
    -d "{\"device_id\": \"g41-full-device\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"$1\"}" \
    | python3 -c 'import json,sys; print(json.load(sys.stdin)["user"]["id"])'
}
ALICE=$(jwt g41-full-alice); BOB=$(jwt g41-full-bob); BOB_U=$(uid g41-full-bob)

# 1. Alice opens a conversation with bob.
C_BODY=$(printf '{"otherParticipants": ["%s"], "title": "Weekly plan"}' "$BOB_U")
CID=$(curl -sf -X POST "${MSC_URL}/v3/messaging/conversations" \
  -H "Authorization: Bearer ${ALICE}" "${KEYS[@]}" \
  -H 'Content-Type: application/json' -d "$C_BODY" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["id"])')
echo "conversation: $CID"

# 2. Alice sends; bob polls until THIS message publishes (match by id).
M_BODY=$(printf '{"conversationId": "%s", "content": "Rotating the flock to the east paddock on Thursday.", "mediaUrls": []}' "$CID")
MID=$(curl -sf -X POST "${MSC_URL}/v3/messaging/messages" \
  -H "Authorization: Bearer ${ALICE}" "${KEYS[@]}" \
  -H 'Content-Type: application/json' -d "$M_BODY" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["id"])')
for i in $(seq 1 24); do
  VISIBLE=$(curl -sf "${MSQ_URL}/v3/messaging/conversations/${CID}/messages?page=1&page_size=5" \
    -H "Authorization: Bearer ${BOB}" "${KEYS[@]}" \
    | python3 -c 'import json,sys
ms = json.load(sys.stdin)
print("yes" if any(m["id"] == "'"$MID"'" for m in ms) else "")')
  [ "$VISIBLE" = yes ] && { echo "published"; break; }
  sleep 5
done
test "$VISIBLE" = yes || { echo "message never published"; exit 1; }

# 3. Bob reacts and reads.
S_BODY=$(printf '{"messageId": "%s", "sentiment": "Positive", "intensity": 0.6}' "$MID")
curl -sf -o /dev/null -X POST "${MSC_URL}/v3/messaging/messages/sentiment" \
  -H "Authorization: Bearer ${BOB}" "${KEYS[@]}" \
  -H 'Content-Type: application/json' -d "$S_BODY" && echo "reacted"
curl -sf -o /dev/null -X POST "${MSC_URL}/v3/messaging/messages/${MID}/read-receipts" \
  -H "Authorization: Bearer ${BOB}" "${KEYS[@]}" && echo "read receipt"

# 4. Alice deletes the message again.
curl -sf -o /dev/null -X DELETE "${MSC_URL}/v3/messaging/messages/${MID}" \
  -H "Authorization: Bearer ${ALICE}" "${KEYS[@]}" && echo "deleted"
```

(`AUTH_URL` is the Auth Service base URL — `http://127.0.0.1:40004` on the
local stack; `MSC_URL` / `MSQ_URL` are the Messaging Command / Query base URLs
— `http://127.0.0.1:40163` / `http://127.0.0.1:40164` locally; `TENANT_ID` is
your tenant's id.)

## Try it live

Open the [Playground](../../playgrounds/playground.html#communication) →
**Communication** — the *Messaging* console exercises the same
create → send → read cycle against the live Messaging services.

## Copy agent prompt

```prompt
You are working with the Inaya Labs Messaging platform. Messaging Command
(writes) is at MSC_URL and Messaging Query (reads) at MSQ_URL; the Auth
Service is at AUTH_URL; the tenant key pair is in INAYA_API_KEY /
INAYA_APP_KEY and TENANT_ID. Obtain three user JWTs via POST
/v3/auth/qa/login {"device_id","locale","tenant_id","username"} with usernames
"g41-agent-a", "g41-agent-b", "g41-agent-c" if tokens are not already set, and
capture each user's id from the login response's user object. Steps: (1) As
agent-a, POST MSC /v3/messaging/conversations {"otherParticipants":
["<agent-b id>"], "title": "agent-demo"} and report the conversation id.
(2) As agent-a, POST MSC /v3/messaging/messages {"conversationId", "content",
"mediaUrls": []} and report the 201 with content null / publishedAt null.
(3) As agent-b, poll MSQ GET /v3/messaging/conversations/{id}/messages
?page=1&page_size=10 every 5 seconds until the message appears, then report
its content and publishedAt. (4) As agent-c (not a participant), POST a message
into the conversation and report the bare 401; also GET the conversation and
report the 4xx. (5) As agent-b, PUT MSC /v3/messaging/messages {"messageId",
"content"} on agent-a's message and report the 403; then as agent-a PUT new
content and report the 200. (6) As agent-b, POST MSC /v3/messaging/messages/
sentiment {"messageId","sentiment":"Positive","intensity":0.7} (200), POST
delivery-receipts then read-receipts (200 each), and read back MSQ GET
/v3/messaging/messages/{id}/receipts, reporting agent-b's status as
DeliveredAndRead. (7) As agent-a, DELETE MSC /v3/messaging/messages/{id} and
report the 204, then repeat and report the 404. Never log the JWTs; report
status codes per step.
```

## API reference

- [Messaging Command — Swagger UI](https://messaging-command-service-sandbox.inayalabs.com/v3/messaging/swagger-ui/) · [Messaging Query — Swagger UI](https://messaging-query-service-sandbox.inayalabs.com/v3/messaging/swagger-ui/)

## Next journeys

- **Publishing and reading a feed** (G-40) — the public, moderated broadcast
  counterpart to these private conversations.
- **Managing users & profiles** (G-30) — the user records every participant
  id maps back to.
- **Authentication & sessions** (G-01) — the token lifecycle that every call
  in this guide depends on.

### G-50 · Transforming audio, images & video
URL: /docs/guides/transforming-media.html

> Trim, transcode, resize, and upscale media with the three transformer services — audio and video as asynchronous jobs (video reports completion over SSE), images as synchronous binary responses — all keyed by local source_path files.

> **Journey.** As a developer, I want to trim an audio clip, convert an image
> to WebP and make a thumbnail, and cut a segment out of a video — without
> standing up my own FFmpeg pipeline — and I want to know reliably when each
> job is done so I can pick up the result.

---

The platform ships three sibling transformer services built on FFmpeg:

| Service | Port (local) | Shape | Operations |
|:---|:---|:---|:---|
| **Audio Transformer** | `:40162` | Async jobs (submit → poll → download) | `trim`, `reverse`, `transcode`, `transmux` |
| **Image Transformer** | `:40030` | Synchronous binary response | `transcode`, `resize`, `upscale` |
| **Video Transformer** | `:40160` | Async jobs (submit → **SSE completion**) | `trim`, `demux`, `mux`, `reverse`, `transcode`, `transmux` |

All three operate on a **`source_path`: a local filesystem path on the host
running the service** — not a URL. There is no upload endpoint and no CDN
involvement: your app (or an upstream step in your pipeline) places the file
on disk — a shared volume, a mounted bucket, a prior job's `result_path` —
and passes the absolute path. Results are written to a per-tenant output
directory on the same host (`$TMPDIR/<service>-output/<tenant_id>/…`), which
makes chaining jobs (audio transcode → trim) trivial.

## Prerequisites

- `ffmpeg` + `ffprobe` on your machine, to fabricate test assets.
- `python3` with Pillow (for format/size assertions), or `file` as a fallback.
- The transformer services authenticate with the **API key pair only**
  (`x-api-key` / `x-client-app-key` headers) — no JWT. The one exception in
  this guide is the Notifications SSE subscribe call, which additionally
  requires a user JWT (see Step 4).
- Sandbox environment accepts any tenant id; the snippets use
  `playground_a43b973c77e94b7f8b0e64d96c86edf8`.

## Concepts

- **Two completion models.** Audio jobs are polled via
  `GET /v3/audio/jobs/{id}` and downloaded via
  `GET /v3/audio/jobs/{id}/download`. Video has **no REST poll or download** —
  completion arrives as a Notifications SSE event (`VideoJobCompleted` /
  `VideoJobFailed`, `sub_topic` = job id) whose payload carries a
  `result_path` you read directly from disk. Images are fully synchronous:
  `POST` returns the transformed bytes (save with `-o`), not JSON.
- **Field-name dialects differ per service.** Audio `trim` uses camelCase
  `inPoint`/`outPoint` (float seconds); video `trim` uses snake_case
  `in_point`/`out_point` as **strings** (`"00:00:05"`); video `transcode`
  takes `resolution` as `"1280:720"` — never `"1080p"`. Images `resize`
  takes `mode: "Single"` (capitalized) plus `width`/`height`.
- **`bitrate` is in kbps.** Audio `transcode` expects e.g. `192`; the service
  appends the `k` when building the FFmpeg argument.
- **Errors are RFC 9457 Problem Details.** Synchronous image failures return
  a `application/problem+json` body. Audio/video job failures surface in the
  job's `error` object (poll) or the SSE `VideoJobFailed` payload.
- **Outputs are transient.** Nothing guarantees how long result files or job
  records live; treat the output directory as a scratch space and copy
  results into your own storage promptly.

## Step 1 — Prepare test assets

Create a workdir with four small, known-good media files. Everything else in
this guide reads from here:

```bash verify=auto
WORKDIR=$(mktemp -d /tmp/g50-media.XXXXXX)
ffmpeg -v error -y -f lavfi -i "sine=frequency=440:duration=8" "${WORKDIR}/tone.wav"
ffmpeg -v error -y -f lavfi -i "testsrc=duration=5:size=640x360:rate=30" \
  -pix_fmt yuv420p "${WORKDIR}/clip.mp4"
python3 -c "from PIL import Image; Image.new('RGB', (1200, 800), (40, 110, 60)).save('${WORKDIR}/photo.png')"
ls -la "${WORKDIR}"
test -s "${WORKDIR}/tone.wav" && test -s "${WORKDIR}/clip.mp4" && test -s "${WORKDIR}/photo.png" \
  && echo "assets ready in ${WORKDIR}"
```

**Expected** — three files: an 8 s WAV tone, a 5 s 640×360 MP4 test pattern,
and a 1200×800 PNG.

## Step 2 — Transcode audio (async: submit → poll → download)

Submit a transcode job (`wav` → `mp3` at 192 kbps), then poll the job until it
completes and download the result:

```bash verify=auto
AUDIO_JOB=$(curl -s -X POST "${AUD_URL}/v3/audio/transcode" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"source_path\": \"${WORKDIR}/tone.wav\", \"target_format\": \"mp3\", \"bitrate\": 192, \"tenant_id\": \"${TENANT_ID}\"}")
echo "${AUDIO_JOB}"
AUDIO_JOB_ID=$(echo "${AUDIO_JOB}" | python3 -c 'import json,sys; print(json.load(sys.stdin)["id"])')
test -n "${AUDIO_JOB_ID}" && echo "job id: ${AUDIO_JOB_ID}"
```

**Expected response** — `202 Accepted` with a `JobResource`:
`{"id":"60OK…","status":"pending","progress":0.0,"filename":"tone.wav",…,"notification_topic":"AudioJobPending",…}`.

Poll until `status` becomes `completed` (a fresh job usually finishes in a
second or two), then download and verify the bytes:

```bash verify=auto
for i in $(seq 1 30); do
  STATUS=$(curl -s "${AUD_URL}/v3/audio/jobs/${AUDIO_JOB_ID}" \
    -H "x-api-key: ${INAYA_API_KEY}" \
    -H "x-client-app-key: ${INAYA_APP_KEY}" \
    | python3 -c 'import json,sys; print(json.load(sys.stdin)["status"])')
  echo "poll ${i}: ${STATUS}"
  [ "${STATUS}" = "completed" ] && break
  [ "${STATUS}" = "failed" ] && { echo "job failed"; exit 1; }
  sleep 2
done
[ "${STATUS}" = "completed" ] || { echo "job did not complete in time"; exit 1; }

curl -sf -o "${WORKDIR}/tone.mp3" \
  "${AUD_URL}/v3/audio/jobs/${AUDIO_JOB_ID}/download" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}"
file "${WORKDIR}/tone.mp3"
file "${WORKDIR}/tone.mp3" | grep -qi "MPEG\|MP3\|Audio" && echo "transcode verified"
```

**Expected** — `status` transitions `pending → processing → completed`; the
download is a real MP3 (`file` reports `Audio file with ID3 … MPEG ADTS …`).
Trimming works the same way with `POST /v3/audio/trim` and camelCase float
seconds: `{"source_path", "inPoint": 2.0, "outPoint": 6.0, "tenant_id"}`.

## Step 3 — Resize and transcode images (sync: bytes come back directly)

Image operations return the transformed file **as the response body**. Save
it with `-o`, then verify with Pillow. Resize the 1200×800 PNG to a 400-wide
thumbnail (JPEG comes back regardless of input format):

```bash verify=auto
curl -s -o "${WORKDIR}/thumb.jpg" -w 'HTTP %{http_code}\n' \
  -X POST "${IMG_URL}/v3/images/resize" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"source_path\": \"${WORKDIR}/photo.png\", \"mode\": \"Single\", \"width\": 400, \"height\": 267, \"tenant_id\": \"${TENANT_ID}\"}"
python3 -c "
from PIL import Image
im = Image.open('${WORKDIR}/thumb.jpg')
assert im.format == 'JPEG' and im.size == (400, 267), (im.format, im.size)
print('resize verified:', im.format, im.size)
"
```

**Expected** — `HTTP 200`, then `resize verified: JPEG (400, 267)`.
`mode` must be exactly `"Single"` (lowercase `single` is rejected), and the
output dimensions follow the `width`/`height` you send.

Now transcode the same PNG to WebP (output format follows `target_format` —
`webp`, `jpeg`, `png`, …):

```bash verify=auto
curl -s -o "${WORKDIR}/photo.webp" -w 'HTTP %{http_code}\n' \
  -X POST "${IMG_URL}/v3/images/transcode" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"source_path\": \"${WORKDIR}/photo.png\", \"target_format\": \"webp\", \"tenant_id\": \"${TENANT_ID}\"}"
python3 -c "
from PIL import Image
im = Image.open('${WORKDIR}/photo.webp')
assert im.format == 'WEBP' and im.size == (1200, 800), (im.format, im.size)
print('transcode verified:', im.format, im.size)
"
```

**Expected** — `HTTP 200`, then `transcode verified: WEBP (1200, 800)`.
For scaling *up* (e.g. 2× super-resolution style upscaling), `POST
/v3/images/upscale` takes an integer `scale_factor` and likewise returns
bytes.

## Step 4 — Trim a video and wait for the SSE completion event

Video has **no REST poll endpoint**. The completion signal is a Notifications
SSE event on topic `VideoJobCompleted` (or `VideoJobFailed`), `sub_topic` =
the job id, whose payload contains `result_path`. Subscribe **before**
submitting, writing the stream to a file:

```bash verify=auto
USER_TOKEN=$(curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"device_id\": \"g50-demo\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"g50-viewer\"}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])')

SSE_FILE=$(mktemp /tmp/g50-sse.XXXXXX)
curl -s -N -X POST "${NOTIF_URL}/v3/notifications/subscribe/multi" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}" \
  -d "{\"tenant_id\": \"${TENANT_ID}\", \"subscriptions\": [{\"topic\": \"VideoJobCompleted\"}, {\"topic\": \"VideoJobFailed\"}]}" \
  > "${SSE_FILE}" 2>&1 &
SSE_PID=$!
sleep 3
echo "subscribed (pid ${SSE_PID}), stream: ${SSE_FILE}"
```

**Expected** — the subscribe call opens an SSE stream (a heartbeat/comment
line arrives within a few seconds). It requires the key pair **and** a user
JWT — transformers themselves need no JWT, only notifications do.

Now submit the trim job. Video bodies are snake_case and time points are
**strings** (`"HH:MM:SS"`):

```bash verify=auto
VIDEO_JOB=$(curl -s -X POST "${VID_URL}/v3/video/trim" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"source_path\": \"${WORKDIR}/clip.mp4\", \"mode\": \"segment\", \"in_point\": \"00:00:01\", \"out_point\": \"00:00:03\", \"tenant_id\": \"${TENANT_ID}\"}")
echo "${VIDEO_JOB}"
VIDEO_JOB_ID=$(echo "${VIDEO_JOB}" | python3 -c 'import json,sys; print(json.load(sys.stdin)["id"])')
test -n "${VIDEO_JOB_ID}" && echo "video job id: ${VIDEO_JOB_ID}"
```

**Expected response** — `202 Accepted` with the same `JobResource` shape as
audio: `{"id":"60OK…","status":"pending",…,"notification_topic":"VideoJobPending","notification_sub_topic":"60OK…"}`.
Note `notification_sub_topic` repeats the job id — that's your SSE filter.

Wait for the completion event, extract `result_path`, and verify the file:

```bash verify=auto
RESULT_PATH=""
for i in $(seq 1 45); do
  if grep -q "VideoJobCompleted" "${SSE_FILE}" 2>/dev/null; then
    RESULT_PATH=$(grep "VideoJobCompleted" "${SSE_FILE}" | head -1 \
      | python3 -c '
import json, sys
line = sys.stdin.read()
data = json.loads(line[line.index("data:") + 5:])
payload = json.loads(data["payload"])
assert data["sub_topic"] == "'"${VIDEO_JOB_ID}"'", data["sub_topic"]
print(payload["result_path"])
')
    [ -n "${RESULT_PATH}" ] && break
  fi
  sleep 2
done
kill "${SSE_PID}" 2>/dev/null
test -n "${RESULT_PATH}" || { echo "no completion event received"; exit 1; }
test -s "${RESULT_PATH}" || { echo "result file missing: ${RESULT_PATH}"; exit 1; }
ffprobe -v error -show_entries format=format_name -of csv=p=0 "${RESULT_PATH}"
echo "video trim verified: ${RESULT_PATH}"
```

**Expected** — within ~30–60 s the SSE stream carries
`data: {"id":"…","topic":"VideoJobCompleted","sub_topic":"<job id>","payload":"{\"job_id\":\"…\",\"result_path\":\"/…/video-transformer-output/<tenant>/<job>.mp4\"}"}`,
the file exists on disk, and `ffprobe` reports `mov,mp4,m4a,3gp,3g2,mj2`.

## Step 5 — Error contract: missing source file

What happens when the input is wrong? For audio, the job is accepted and then
fails asynchronously — the failure is in the polled job record, not the
submit response:

```bash verify=auto
BAD_JOB=$(curl -s -X POST "${AUD_URL}/v3/audio/transcode" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"source_path\": \"${WORKDIR}/does-not-exist.wav\", \"target_format\": \"mp3\", \"bitrate\": 128, \"tenant_id\": \"${TENANT_ID}\"}")
BAD_JOB_ID=$(echo "${BAD_JOB}" | python3 -c 'import json,sys; print(json.load(sys.stdin)["id"])')
echo "accepted bad-source job: ${BAD_JOB_ID}"

for i in $(seq 1 15); do
  JOB_JSON=$(curl -s "${AUD_URL}/v3/audio/jobs/${BAD_JOB_ID}" \
    -H "x-api-key: ${INAYA_API_KEY}" \
    -H "x-client-app-key: ${INAYA_APP_KEY}")
  echo "${JOB_JSON}" | python3 -c 'import json,sys; print("status:", json.load(sys.stdin)["status"])'
  echo "${JOB_JSON}" | grep -q '"status":"failed"' && break
  sleep 2
done
echo "${JOB_JSON}" | python3 -c '
import json, sys
job = json.load(sys.stdin)
assert job["status"] == "failed", job["status"]
assert "not found" in json.dumps(job.get("error", {})).lower(), job.get("error")
print("async failure surfaced in job record")
'
```

**Expected** — the submit returns `202` (the path isn't validated up front);
after a poll or two, `status` flips to `failed` and the job's `error` detail
explains the source file was not found. For images (synchronous), the same
mistake returns `400` immediately with an RFC 9457 `application/problem+json`
body.

## Putting it together

A single script that walks all three transformers end to end (assumes
`WORKDIR` assets from Step 1 and `USER_TOKEN` from Step 4 exist):

```bash
# 1. Audio: transcode wav → mp3 192k, poll, download
JOB=$(curl -sf -X POST "${AUD_URL}/v3/audio/transcode" -H "Content-Type: application/json" \
  "${KEYS[@]}" -d "{\"source_path\": \"${WORKDIR}/tone.wav\", \"target_format\": \"mp3\", \"bitrate\": 192, \"tenant_id\": \"${TENANT_ID}\"}")
ID=$(echo "$JOB" | python3 -c 'import json,sys; print(json.load(sys.stdin)["id"])')
until curl -s "${AUD_URL}/v3/audio/jobs/${ID}" "${KEYS[@]}" | grep -q '"status":"completed"'; do sleep 2; done
curl -sf -o "${WORKDIR}/tone.mp3" "${AUD_URL}/v3/audio/jobs/${ID}/download" "${KEYS[@]}"

# 2. Image: 400x267 thumbnail, bytes returned directly
curl -sf -o "${WORKDIR}/thumb.jpg" -X POST "${IMG_URL}/v3/images/resize" \
  -H "Content-Type: application/json" "${KEYS[@]}" \
  -d "{\"source_path\": \"${WORKDIR}/photo.png\", \"mode\": \"Single\", \"width\": 400, \"height\": 267, \"tenant_id\": \"${TENANT_ID}\"}"

# 3. Video: trim, completion via SSE, read result_path from disk
curl -sf -N -X POST "${NOTIF_URL}/v3/notifications/subscribe/multi" \
  -H "Authorization: Bearer ${USER_TOKEN}" -H "Content-Type: application/json" "${KEYS[@]}" \
  -d "{\"tenant_id\": \"${TENANT_ID}\", \"subscriptions\": [{\"topic\": \"VideoJobCompleted\"}]}" \
  > /tmp/g50-sse.txt &
curl -sf -X POST "${VID_URL}/v3/video/trim" -H "Content-Type: application/json" "${KEYS[@]}" \
  -d "{\"source_path\": \"${WORKDIR}/clip.mp4\", \"mode\": \"segment\", \"in_point\": \"00:00:01\", \"out_point\": \"00:00:03\", \"tenant_id\": \"${TENANT_ID}\"}" \
  | python3 -c 'import json,sys; print("video job:", json.load(sys.stdin)["id"])'
until grep -q VideoJobCompleted /tmp/g50-sse.txt 2>/dev/null; do sleep 2; done
grep VideoJobCompleted /tmp/g50-sse.txt | head -1
```

(`KEYS[@]` is `-H "x-api-key: …" -H "x-client-app-key: …"`;
`AUD_URL`/`IMG_URL`/`VID_URL`/`NOTIF_URL` are the transformer/notification
base URLs — `http://127.0.0.1:40162` / `:40030` / `:40160` / `:40013`
locally; `AUTH_URL` is `http://127.0.0.1:40004`; `TENANT_ID` is your tenant.)

## Try it live

Open the [Playground](../../playgrounds/playground.html#media) → **Media** —
the *Audio / Image / Video Transformer* consoles exercise the same
operations against the live services.

## Copy agent prompt

```prompt
You are working with the Inaya Labs media transformers. Audio Transformer is
at AUD_URL (:40162), Image Transformer at IMG_URL (:40030), Video
Transformer at VID_URL (:40160), Notifications at NOTIF_URL (:40013), Auth at
AUTH_URL (:40004); the tenant key pair is in INAYA_API_KEY / INAYA_APP_KEY and
TENANT_ID. Transformers authenticate with the key pair ONLY (no JWT), except
the notifications subscribe call which also needs a user JWT from POST
AUTH /v3/auth/qa/login {"device_id","locale":"en-US","tenant_id","username"}.
source_path is always a LOCAL FILESYSTEM PATH on the service host — fabricate
test assets with ffmpeg/Pillow in a fresh workdir first. Steps: (1) Create an
8 s sine WAV, a 5 s 640x360 testsrc MP4, and a 1200x800 PNG. (2) Audio:
POST AUD /v3/audio/transcode {"source_path","target_format":"mp3","bitrate":
192,"tenant_id"} — 202 JobResource; poll GET /v3/audio/jobs/{id} until
"completed"; download GET /v3/audio/jobs/{id}/download and verify with `file`
that it is an MP3. (3) Image: POST IMG /v3/images/resize {"source_path",
"mode":"Single","width":400,"height":267,"tenant_id"} — the RESPONSE BODY IS
THE FILE; save with -o and assert JPEG 400x267 with Pillow. Then transcode to
webp and assert WEBP. (4) Video: FIRST subscribe to SSE POST NOTIF
/v3/notifications/subscribe/multi {"tenant_id","subscriptions":[{"topic":
"VideoJobCompleted"},{"topic":"VideoJobFailed"}]} in the background writing
to a file (needs the JWT), THEN POST VID /v3/video/trim {"source_path",
"mode":"segment","in_point":"00:00:01","out_point":"00:00:03","tenant_id"} —
202 JobResource. Grep the SSE file for VideoJobCompleted with sub_topic equal
to the job id, parse payload.result_path, assert the file exists and ffprobe
reports an mp4. (5) Error contract: submit an audio transcode with a
nonexistent source_path — expect 202, then a polled status "failed" whose
error detail mentions the file not being found; the equivalent image call
fails synchronously with 400 problem+json. Report status codes and byte
counts per step; never log the JWT.
```

## API reference

- [Audio Transformer — Swagger UI](https://audio-transformer-service-sandbox.inayalabs.com/v3/swagger-ui/) · [Image Transformer — Swagger UI](https://image-transformer-service-sandbox.inayalabs.com/v3/swagger-ui/) · [Video Transformer — Swagger UI](https://video-transformer-service-sandbox.inayalabs.com/v3/swagger-ui/)

## Next journeys

- **Scrubbing PII from text** (G-51) — clean transcripts and captions before they leave your perimeter.
- **Uploading files** (G-02) — the upload flow that can place source files on the service host for you.
- **Errors, retries & idempotency** (G-04) — job polling and retry strategies in detail.

### G-51 · Scrubbing PII from text
URL: /docs/guides/scrubbing-pii-from-text.html

> Redact emails, phones, credit cards, IPs, and URLs into stable placeholders before text leaves your perimeter, then deanonymize later to restore the originals — structural detection that works identically across languages.

> **Journey.** As a developer, I want to send user text through an LLM or an
> analytics pipeline without leaking anybody's contact details — and get the
> original text back afterwards, so my app can show the user *their* data
> while the outside world only ever saw placeholders.

---

The **Privacy Scrubbing service** (`:40052`, prefix `/v3`) is a deliberately
small, self-contained filter: it detects **five structural PII types** —
`EMAIL`, `PHONE`, `CREDIT_CARD`, `IP_ADDRESS`, `URL` — and replaces each
match with a stable placeholder (`[EMAIL_1]`, `[PHONE_2]`, …). A companion
`deanonymize` call restores the originals later.

Two design facts shape everything else:

- **The REST API is intentionally anonymous.** Scrub and deanonymize sit in
  the no-auth surface — the service never touches tenant data, and the text
  you send never leaves the process (no database, no downstream calls).
  (The MCP stream is the exception: it is key-pair + JWT gated like every
  other agent surface.)
- **The placeholder→original mapping lives server-side in an in-memory
  vault.** Scrubbing stores each minted placeholder in process memory;
  deanonymize looks it up. That makes the round trip trivial — you send
  only the scrubbed text back, no mapping table to manage — but it means
  the mapping is **per service instance and lost on restart**, and a
  placeholder minted twice maps to the *latest* value. For durable
  round trips, keep the `findings` array from the scrub response
  (it carries `value` + `placeholder` client-side) and restore yourself,
  or accept the vault's lifetime.

## Prerequisites

- Just the service itself — **no other stack component is required** (pure
  regex engine; no Postgres, no Evergreen AI, no models).
- Nothing to authenticate on REST: no API keys, no JWT.
- Body limit is 2 MiB per request.

## Concepts

- **Detection is structural, not lexical.** Patterns match shape, not
  vocabulary — an email is `@`-and-dots, a phone is a structured digit run,
  a card is 13–19 digits that pass a **Luhn checksum** (so order IDs and
  tracking numbers don't false-positive). No names, street addresses, or ID
  numbers are detected — "Amina" in the examples below passes through.
- **Overlap resolution is greedy by priority.** When detectors overlap
  (an email inside a URL, a phone inside a credit card), the match is
  redacted once under the higher-priority kind: `CREDIT_CARD` > `URL` >
  `EMAIL` > `IP_ADDRESS` > `PHONE`.
- **Counters are per kind.** The `n` in `[EMAIL_n]` counts emails only;
  the first phone is always `[PHONE_1]` regardless of how many emails
  preceded it.
- **Language independence.** Because detection is structural, surrounding
  text in any script — Arabic, Cyrillic, CJK — is untouched while the PII
  inside it is still caught. The optional `locale` field is currently
  **accepted and ignored**; there are no locale-specific patterns yet.
- **Errors are minimal JSON.** Empty/whitespace text → `422`
  `{"error":"text must not be empty"}`.

## Step 1 — Scrub a message

```bash verify=auto
SCRUB_JSON=$(curl -s -X POST "${SCRUB_URL}/v3/scrub" \
  -H "Content-Type: application/json" \
  -d '{"text": "Contact Amina at amina@example.com or +968 9000 1234; card 4242 4242 4242 4242, ip 192.168.10.1, see https://user.example.com/path", "locale": "en"}')
echo "${SCRUB_JSON}" | python3 -c '
import json, sys
r = json.load(sys.stdin)
print("scrubbed:", r["scrubbed_text"])
for f in r["findings"]:
    print("  {p:>18}  <- {k}  {v!r}  [{s}:{e}]".format(
        p=f["placeholder"], k=f["kind"], v=f["value"], s=f["start"], e=f["end"]))
'
SCRUBBED_TEXT=$(echo "${SCRUB_JSON}" | python3 -c 'import json,sys; print(json.load(sys.stdin)["scrubbed_text"])')
echo "${SCRUBBED_TEXT}" | grep -q "\[EMAIL_1\]" && echo "email redacted"
```

**Expected response** — `200` with `scrubbed_text` (the five kinds each
redacted once) and a `findings` array: `{kind, value, placeholder, start,
end}` with **byte offsets** into the original text. The `locale` field is
optional and currently ignored.

## Step 2 — Deanonymize to restore the original

Send the scrubbed text back; the server-side vault restores the originals:

```bash verify=auto
curl -s -X POST "${SCRUB_URL}/v3/deanonymize" \
  -H "Content-Type: application/json" \
  -d "{\"text\": \"${SCRUBBED_TEXT}\"}" \
| python3 -c '
import json, sys
r = json.load(sys.stdin)
original = "Contact Amina at amina@example.com or +968 9000 1234; card 4242 4242 4242 4242, ip 192.168.10.1, see https://user.example.com/path"
assert r["deanonymized_text"] == original, r["deanonymized_text"]
assert r["restored_count"] == 5, r["restored_count"]
print("round trip restored", r["restored_count"], "entities verbatim")
'
```

**Expected** — `200` with the original text byte-for-byte and
`restored_count: 5`. Placeholders the vault doesn't know pass through
untouched (and don't count) — try `"call [PHONE_9] now"` →
`restored_count: 0`.

## Step 3 — Language independence

Detection keys on structure, so it works identically whatever language the
human text around the PII is written in — the `locale` field changes
nothing today:

```bash verify=auto
curl -s -X POST "${SCRUB_URL}/v3/scrub" \
  -H "Content-Type: application/json" \
  -d '{"text": "تواصل مع آمنة عبر amina@example.com أو الرقم +968 9000 1234"}' \
| python3 -c '
import json, sys
r = json.load(sys.stdin)
assert r["scrubbed_text"] == "تواصل مع آمنة عبر [EMAIL_1] أو الرقم [PHONE_1]", r["scrubbed_text"]
kinds = [f["kind"] for f in r["findings"]]
assert kinds == ["EMAIL", "PHONE"], kinds
print("arabic text preserved, PII caught:", r["scrubbed_text"])
'
```

**Expected** — the Arabic prose survives untouched while the email and
phone inside it are redacted — the same result the English text produced.
Two honest caveats: proper names are **not** detected (structural types
only), and this is *script-agnostic* detection, not multilingual entity
recognition — locale-specific refinements are a planned future layer.

## Step 4 — Precision rules worth knowing

```bash verify=auto
# Luhn guard: 13-19 digit runs that are NOT valid cards pass through
curl -s -X POST "${SCRUB_URL}/v3/scrub" -H "Content-Type: application/json" \
  -d '{"text": "order 4242 4242 4242 4243 shipped"}' \
| python3 -c '
import json, sys
r = json.load(sys.stdin)
assert r["findings"] == [], r["findings"]
print("luhn guard ok: invalid card number left alone")
'

# Overlap: an email inside a URL is redacted ONCE, as the URL
curl -s -X POST "${SCRUB_URL}/v3/scrub" -H "Content-Type: application/json" \
  -d '{"text": "see https://user@example.com/path now"}' \
| python3 -c '
import json, sys
r = json.load(sys.stdin)
assert r["scrubbed_text"] == "see [URL_1] now", r["scrubbed_text"]
assert [f["kind"] for f in r["findings"]] == ["URL"]
print("overlap rule ok: redacted once as", r["findings"][0]["placeholder"])
'
```

**Expected** — invalid card numbers survive (Luhn checksum), and nested
matches collapse to a single redaction under the higher-priority kind.

## Step 5 — Error contract

```bash verify=auto
# 422 — empty / whitespace-only text
CODE=$(curl -s -o /tmp/g51-err.txt -w '%{http_code}' -X POST "${SCRUB_URL}/v3/scrub" \
  -H "Content-Type: application/json" \
  -d '{"text": "   "}')
[ "${CODE}" = "422" ] && grep -q "text must not be empty" /tmp/g51-err.txt \
  && echo "422 validation error ok: $(cat /tmp/g51-err.txt)"
```

**Expected** — `422` with `{"error":"text must not be empty"}` (a missing
`text` field deserializes to the same validation failure).

## Putting it together

A two-phase flow: scrub before sending text to a third party, deanonymize
when displaying results back to the user:

```bash
# Phase 1 — before the text leaves your perimeter
RESP=$(curl -sf -X POST "$SCRUB_URL/v3/scrub" -H "Content-Type: application/json" \
  -d '{"text": "Reply to john.doe@company.com from 203.0.113.7 please"}')
SAFE=$(echo "$RESP" | python3 -c 'import json,sys; print(json.load(sys.stdin)["scrubbed_text"])')
# … send $SAFE to the LLM / analytics / support tool …

# Phase 2 — restore for the user’s own view
curl -sf -X POST "$SCRUB_URL/v3/deanonymize" -H "Content-Type: application/json" \
  -d "{\"text\": \"$SAFE\"}"
```

(`SCRUB_URL` is `http://127.0.0.1:40052` on the local stack. No auth
headers are needed — or accepted — on these two endpoints.)

## Try it live

Open the [Playground](../../playgrounds/playground.html#media) → **Media**
— the *Privacy Scrubbing* console exercises the same scrub → deanonymize
round trip against the live service.

## Copy agent prompt

```prompt
You are working with the Inaya Labs Privacy Scrubbing service at SCRUB_URL
(:40052, prefix /v3). The REST endpoints are ANONYMOUS by design — no
x-api-key, no JWT. Steps: (1) POST /v3/scrub {"text": "Contact Amina at
amina@example.com or +968 9000 1234; card 4242 4242 4242 4242, ip
192.168.10.1, see https://user.example.com/path", "locale": "en"} — assert
200, capture scrubbed_text, and verify findings[] entries carry kind
(EMAIL/PHONE/CREDIT_CARD/IP_ADDRESS/URL), value, placeholder ([EMAIL_1]
style), and byte-offset start/end. (2) POST /v3/deanonymize {"text":
<scrubbed_text>} — assert the response equals the ORIGINAL text verbatim
and restored_count is 5. Note the mapping lives in the service's in-memory
vault: per-instance, lost on restart, latest-wins on placeholder reuse —
for durable round trips keep the findings array and restore client-side.
(3) Language independence: scrub "تواصل مع آمنة عبر amina@example.com أو
الرقم +968 9000 1234" — assert the Arabic prose is untouched while the
email/phone are redacted; note locale is accepted but IGNORED and proper
names are NOT detected. (4) Precision: "order 4242 4242 4242 4243 shipped"
→ 0 findings (Luhn guard); "see https://user@example.com/path now" → one
[URL_1] redaction (overlap resolution, CREDIT_CARD > URL > EMAIL >
IP_ADDRESS > PHONE). (5) Errors: {"text": " "} → 422 {"error":"text must
not be empty"}. Report findings counts and restored_count per step. The
MCP tool scrub_sensitive_data exists but is key+JWT gated, unlike the REST
surface.
```

## API reference

- [Privacy Scrubbing — Swagger UI](https://privacy-scrubbing-service-sandbox.inayalabs.com/v3/swagger-ui/)

## Next journeys

- **Transforming audio, images & video** (G-50) — the rest of the media family.
- **Caching with Memcache** (G-70) — cache scrub results keyed by content hash.
- **Errors, retries & idempotency** (G-04) — designing around the in-memory vault's lifetime.

### G-60 · Translating text with Linga
URL: /docs/guides/translating-text-with-linga.html

> Translate single texts between BCP-47 locales with optional domain jargon and generational vernacular, auto-detect the source language, or run a bidirectional language-pair channel that flips direction on its own — all through one REST endpoint pair fronting Evergreen AI.

> **Journey.** As a developer, I want my app to translate user text between
> languages — with the right domain vocabulary and tone — and I want a
> bilingual chat feature where I configure the pair once and the service
> figures out which way to translate on every message.

---

**Linga** (`:40065`, prefix `/v3/translation`) is a thin, prompt-engineered
front end to Evergreen AI. It holds no translation models of its own: each
call opens a single-turn Evergreen session (Llama 3.2, text-only), sends a
carefully constructed translator system prompt plus your text, and parses the
structured JSON the model is asked to return. That means translation quality
follows the models Evergreen serves, and Linga calls inherit Evergreen's
availability.

There are exactly two REST endpoints:

| Endpoint | Purpose |
|:---|:---|
| `POST /v3/translation/translate` | One-shot translation with explicit source/target (or auto-detected source) |
| `POST /v3/translation/translate-bidirectional` | A configured language pair that detects the input's language and flips direction automatically |

Both authenticate with the **tenant key pair *and* a user JWT**
(`x-api-key`, `x-client-app-key`, `Authorization: Bearer …`). The JWT is
forwarded to Evergreen AI for the backend session — in non-sandbox
environments it must be a valid platform JWT (the QA-login flow below
provides one); it is not optional even though Linga itself is only proxying.

## Prerequisites

- Auth Service (for a user JWT) and Evergreen AI running and reachable; the
  QA tenant's key pair in `INAYA_API_KEY` / `INAYA_APP_KEY`.
- An Evergreen model capable of the requested language pair. Linga demands
  Llama 3.2 (text-only) — if Evergreen has no capacity for it, calls fail
  with `500` after an internal retry window (~75 s).
- Linga's OpenAPI spec (`GET /v3/translation/api-docs/openapi.json`) is the
  discovery surface for the `jargon` / `vernacular` enums — there is no
  language-list endpoint.

## Concepts

- **`LanguageDescription`** — every language slot is an object of up to three
  optional fields, all snake_case: `locale` (BCP-47, e.g. `"es-ES"`),
  `jargon` (domain vocabulary; 15 PascalCase values such as `Agriculture`,
  `Legal`, `Healthcare`, `LayPerson`), and `vernacular` (generational tone;
  7 values such as `GenerationZ`, `Millennials`, `Neutral`). Any subset
  composes: `{"locale": "es-ES", "jargon": "Agriculture"}` means *European
  Spanish using farming vocabulary*. An entirely empty `{}` means
  *auto-detect* (valid for `source`; for `target` it is rejected).
- **Response echoes the *configured* languages, not the detected ones.** In
  `translate`, `source_language` / `target_language` repeat your
  configuration verbatim (jargon included, e.g. `"es-ES, Agriculture"`) —
  with auto-detect source, `source_language` comes back empty. Use
  `translate-bidirectional` when you need the model's detected locale: it
  returns a separate `detected_source_locale` (BCP-47) plus human-readable
  `source_language` / `target_language` names.
- **Errors are plain text, not JSON.** Business-rule failures (`400`) return
  a bare text body (`Invalid request: No target language specified`); the
  401 auth failure returns a small JSON error object; `422`
  (`DetectedLanguageNotInPair`) is a best-effort prompt contract — the model
  is instructed to reply `LANGUAGE_NOT_IN_PAIR` for out-of-pair input, but
  compliance is model-dependent, so don't build hard control flow on it.
- **Translation is per-call and stateless.** There is no glossary, memory, or
  feedback loop; consistency across calls comes from the system prompt, not
  from retained state.

## Step 1 — Get a user JWT

Both endpoints require the tenant key pair **and** a user JWT (forwarded to
Evergreen). Mint one via the Auth Service QA login:

```bash verify=auto
USER_TOKEN=$(curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"device_id\": \"g60-demo\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"g60-translator\"}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])')
test -n "${USER_TOKEN}" && echo "JWT acquired (${#USER_TOKEN} chars)"
```

**Expected** — a long PS512-signed JWT; `qa/login` requires `device_id`
(min 8 chars), `locale`, `tenant_id`, and `username`.

## Step 2 — One-shot translation with explicit locales

```bash verify=auto
curl -s -X POST "${LIN_URL}/v3/translation/translate" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}" \
  -d '{"source": {"locale": "en-US"}, "target": {"locale": "es-ES"}, "text": "The chickens escaped again."}' \
| python3 -c '
import json, sys
r = json.load(sys.stdin)
assert r["original_text"] == "The chickens escaped again."
assert r["source_language"] == "en-US" and r["target_language"] == "es-ES"
assert len(r["translated_text"]) > 0
print("translated:", r["translated_text"])
'
```

**Expected response** — `200` with snake_case fields:
`{"original_text", "translated_text", "source_language": "en-US", "target_language": "es-ES"}`.
`text` accepts 1–10,000,000 characters; an empty `target` is rejected.

## Step 3 — Auto-detect the source language

Send an empty `source` `{}` and Linga asks the model to identify the language:

```bash verify=auto
curl -s -X POST "${LIN_URL}/v3/translation/translate" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}" \
  -d '{"source": {}, "target": {"locale": "en-US"}, "text": "Buenos días, ¿cómo estás?"}' \
| python3 -c '
import json, sys
r = json.load(sys.stdin)
assert r["target_language"] == "en-US"
assert "morning" in r["translated_text"].lower() or "how are you" in r["translated_text"].lower()
print("auto-detected translation:", r["translated_text"])
print("note: source_language echoes your config, so it is empty:", repr(r["source_language"]))
'
```

**Expected** — `200` with an English translation. Note the quirk: with
auto-detect, `source_language` echoes your (empty) configuration rather than
the detected locale — use bidirectional mode (Step 4) if you need the
detected locale reported.

## Step 4 — Domain jargon and generational vernacular

Compose `locale` + `jargon` + `vernacular` to steer vocabulary and tone.
Jargon values are PascalCase enum variants (`Agriculture`, `Construction`,
`Education`, `Engineering`, `Entertainment`, `Finance`, `Healthcare`,
`InformationTechnologyAndTelecom`, `LayPerson`, `Legal`, `MarketingAndPr`,
`Manufacturing`, `NewsAndJournalism`, `Property`, `Retail`); vernacular
values are `BabyBoomers`, `GenerationX`, `Millennials`, `GenerationZ`,
`GenerationAlpha`, `TheSilentGeneration`, `Neutral`:

```bash verify=auto
curl -s -X POST "${LIN_URL}/v3/translation/translate" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}" \
  -d '{"source": {"locale": "en-US"}, "target": {"locale": "es-ES", "jargon": "Agriculture"}, "text": "The heifer is due to calve next week."}' \
| python3 -c '
import json, sys
r = json.load(sys.stdin)
assert "Agriculture" in r["target_language"], r["target_language"]
assert len(r["translated_text"]) > 0
print("jargon-steered:", r["target_language"], "->", r["translated_text"])
'
```

**Expected** — `200`; the configured jargon is echoed back inside
`target_language` (`"es-ES, Agriculture"`), confirming the steer was applied.
(The model is also instructed to add disclaimer suffixes when translating
to/from legal, medical, or safety jargon.)

## Step 5 — Bidirectional language pair (auto-flipping)

For bilingual UX (support chat, marketplace messaging), configure the pair
once; Linga detects which language each input is in and translates to the
other. `language_a` and `language_b` must each carry at least one field:

```bash verify=auto
PAIR_A=$(curl -s -X POST "${LIN_URL}/v3/translation/translate-bidirectional" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}" \
  -d '{"language_a": {"locale": "en-US"}, "language_b": {"locale": "es-ES"}, "text": "Good morning, how are the cows?"}')
echo "${PAIR_A}"
echo "${PAIR_A}" | python3 -c '
import json, sys
r = json.load(sys.stdin)
assert r["detected_source_locale"] == "en-US", r
assert "vaca" in r["translated_text"].lower() or "ganado" in r["translated_text"].lower(), r["translated_text"]
print("A->B ok")
'
```

**Expected response** — `200` with `original_text`, `translated_text`,
`source_language` / `target_language` (human-readable names this time, e.g.
`"English"` / `"Spanish"`), and `detected_source_locale: "en-US"`.

Now send Spanish down the same pair — the direction flips with no code change:

```bash verify=auto
PAIR_B=$(curl -s -X POST "${LIN_URL}/v3/translation/translate-bidirectional" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}" \
  -d '{"language_a": {"locale": "en-US"}, "language_b": {"locale": "es-ES"}, "text": "Buenos días, ¿cómo están las vacas?"}')
echo "${PAIR_B}"
echo "${PAIR_B}" | python3 -c '
import json, sys
r = json.load(sys.stdin)
assert r["detected_source_locale"] == "es-ES", r
assert "cow" in r["translated_text"].lower(), r["translated_text"]
print("B->A ok: direction flipped automatically")
'
```

**Expected** — `200` with `detected_source_locale: "es-ES"` and an English
translation. If the input matches neither configured language, the model is
asked to reply `LANGUAGE_NOT_IN_PAIR`, which surfaces as `422` — a
best-effort contract that depends on model compliance, so treat it as a
graceful-degradation path rather than a guarantee.

## Step 6 — Error contract

Missing target, missing key pair, and a short `device_id` on QA login each
fail distinctly:

```bash verify=auto
# 400 plain text — empty target
CODE=$(curl -s -o /tmp/g60-err1.txt -w '%{http_code}' -X POST "${LIN_URL}/v3/translation/translate" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}" \
  -d '{"source": {"locale": "en-US"}, "target": {}, "text": "hello"}')
[ "${CODE}" = "400" ] && grep -q "No target language specified" /tmp/g60-err1.txt \
  && echo "400 plain-text error ok: $(cat /tmp/g60-err1.txt)"

# 401 JSON — key pair omitted
CODE=$(curl -s -o /tmp/g60-err2.txt -w '%{http_code}' -X POST "${LIN_URL}/v3/translation/translate" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${USER_TOKEN}" \
  -d '{"source": {}, "target": {"locale": "en-US"}, "text": "hola"}')
[ "${CODE}" = "401" ] && grep -q "Missing Tenant API Key" /tmp/g60-err2.txt \
  && echo "401 auth error ok: $(cat /tmp/g60-err2.txt)"
```

**Expected** — `400` with a **plain-text** body (`Invalid request: No target
language specified`) and `401` with a JSON body. Note the asymmetry: not all
errors are Problem Details JSON — parse by status code first, then inspect
the body format.

## Putting it together

A single script that runs the whole journey (assumes `USER_TOKEN` from
Step 1):

```bash
KEYS=(-H "x-api-key: $INAYA_API_KEY" -H "x-client-app-key: $INAYA_APP_KEY")

# 1. One-shot, explicit locales
curl -sf -X POST "$LIN_URL/v3/translation/translate" -H "Authorization: Bearer $USER_TOKEN" \
  "${KEYS[@]}" -H "Content-Type: application/json" \
  -d '{"source": {"locale": "en-US"}, "target": {"locale": "de-DE"}, "text": "The pump is running."}'

# 2. Auto-detected source
curl -sf -X POST "$LIN_URL/v3/translation/translate" -H "Authorization: Bearer $USER_TOKEN" \
  "${KEYS[@]}" -H "Content-Type: application/json" \
  -d '{"source": {}, "target": {"locale": "en-US"}, "text": "Guten Morgen."}'

# 3. Domain jargon
curl -sf -X POST "$LIN_URL/v3/translation/translate" -H "Authorization: Bearer $USER_TOKEN" \
  "${KEYS[@]}" -H "Content-Type: application/json" \
  -d '{"source": {"locale": "en-US"}, "target": {"locale": "es-ES", "jargon": "Agriculture"}, "text": "Rotate the pasture before overgrazing."}'

# 4. Bidirectional pair — two messages, two directions, same config
curl -sf -X POST "$LIN_URL/v3/translation/translate-bidirectional" -H "Authorization: Bearer $USER_TOKEN" \
  "${KEYS[@]}" -H "Content-Type: application/json" \
  -d '{"language_a": {"locale": "en-US"}, "language_b": {"locale": "es-ES"}, "text": "See you at the market."}'
curl -sf -X POST "$LIN_URL/v3/translation/translate-bidirectional" -H "Authorization: Bearer $USER_TOKEN" \
  "${KEYS[@]}" -H "Content-Type: application/json" \
  -d '{"language_a": {"locale": "en-US"}, "language_b": {"locale": "es-ES"}, "text": "Nos vemos en el mercado."}'
```

(`LIN_URL` is the Linga base URL — `http://127.0.0.1:40065` on the local
stack; `AUTH_URL` is `http://127.0.0.1:40004`; `TENANT_ID` is your tenant.)

## Try it live

Open the [Playground](../../playgrounds/playground.html#translation) →
**Translation & Voice Cloning** — the *Linga* console exercises the same
translate calls against the live service.

## Copy agent prompt

```prompt
You are working with the Inaya Labs Linga translation service at LIN_URL
(:40065, prefix /v3/translation), backed by Evergreen AI. The tenant key pair
is in INAYA_API_KEY / INAYA_APP_KEY, TENANT_ID holds the tenant, and AUTH_URL
(:40004) is the Auth Service. Every call needs BOTH the key-pair headers AND
a user JWT (obtain via POST AUTH /v3/auth/qa/login {"device_id" (min 8 chars),
"locale": "en-US", "tenant_id", "username"}; the JWT is forwarded to
Evergreen, so it must be valid). Steps: (1) Translate POST LIN
/v3/translation/translate {"source": {"locale": "en-US"}, "target": {"locale":
"es-ES"}, "text": "The chickens escaped again."} — assert 200 and snake_case
fields original_text/translated_text/source_language/target_language. (2)
Auto-detect: same endpoint with "source": {} and Spanish text — assert 200;
note source_language echoes the (empty) config. (3) Jargon: add
"jargon": "Agriculture" to the target and assert the response
target_language contains "Agriculture". (4) Bidirectional: POST
/v3/translation/translate-bidirectional {"language_a": {"locale": "en-US"},
"language_b": {"locale": "es-ES"}, "text": ...} first with English, then with
Spanish — assert detected_source_locale flips en-US -> es-ES and the
translations go the opposite ways. (5) Errors: empty target -> 400 PLAIN
TEXT "Invalid request: No target language specified"; omitting the key pair
-> 401 JSON. Note there is no batch endpoint and no language-list endpoint —
the jargon/vernacular enums are documented in the OpenAPI JSON at
/v3/translation/api-docs/openapi.json. Never log the JWT; report status codes
and translated_text per step.
```

## API reference

- [Linga — Swagger UI](https://linga-service-sandbox.inayalabs.com/v3/translation/swagger-ui/)

## Next journeys

- **Cloning a voice and speaking** (G-61) — register a VoicePrint and generate speech on the Speech service.
- **Your first AI chat session** (G-10) — the Evergreen session mechanics Linga uses under the hood.
- **Errors, retries & idempotency** (G-04) — Linga's internal Evergreen retry window, and what a 500 means here.

### G-61 · Cloning a voice and speaking
URL: /docs/guides/cloning-a-voice-and-speaking.html

> Register a VoicePrint from a microphone recording (pre-recorded or consent-script verified), then synthesize speech in that voice — as binary MP3/WAV or as an SSE stream of PCM chunks — on the Speech service's local Qwen3-TTS model.

> **Journey.** As a developer, I want my app to let a user register their
> voice once, then speak any text in that voice — with a consent-verified
> option for trusted, anti-spoof enrollment — and receive audio my client
> can play or stream.

---

The **Speech service** (`:40161`, prefix `/v3`) runs a **local Qwen3-TTS
voice-cloning model** (MLX on macOS) — no cloud TTS API key, no per-call
vendor cost. Voice cloning is a two-resource dance:

1. A **VoicePrint** — a 24 kHz mono WAV of the user's voice, uploaded as
   multipart audio, Whisper-checked for intelligible speech, stored in the
   **CDN**. Registration returns a *path*, `voice_print_wav_path`, which you
   pass back as `reference_voiceprint_path` on every synthesis call.
2. **Speech synthesis** — `POST /v3/text-to-speech` clones the reference
   voice and returns **binary audio** (MP3 by default), or an **SSE stream**
   of base64 PCM chunks when `stream=true`.

Everything is snake_case; field names are long and literal
(`reference_voiceprint_path`, `voice_print_wav_path`).

## Prerequisites

- The Speech service boots only when its **local models** are present:
  Qwen3-TTS (`QWEN_TTS_MODEL_DIR`, default
  `~/.InayaLabs/models/hf/Qwen3-TTS-12Hz-1.7B-Base`) and Whisper-tiny
  (`WHISPER_TINY_MODEL_DIR`). **ffmpeg** must be on `PATH`.
- **CDN** (voice-print storage) must be running; the **verified** enrollment
  flow additionally needs **Memcache** (script TTL storage) and **Evergreen
  AI** (script generation). The unverified flow needs only the CDN.
- Auth is the standard pair: `x-api-key` + `x-client-app-key` **and** a user
  JWT, which the service forwards to the CDN when downloading your
  voice print. On macOS you can fabricate a test recording with
  `say -o sample.wav --data-format=LEI16@24000 "…"`.
- One voice print **per user**: re-registering overwrites the previous one.
  There are no list/get/delete endpoints; the path embeds
  `sha256(JWT subject)`, so it is deterministic per user.

## Concepts

- **Two enrollment flows.**
  *Unverified* (`POST /v3/voice-prints`): one multipart upload, single step,
  right for low-trust or prototyping scenarios. *Verified*
  (two steps: `POST /v3/verified-voice-prints/verifiable-paragraphs` → read
  the returned script aloud → `POST /v3/verified-voice-prints/{registration_id}`
  with the recording): the service LLM-generates a fresh consent script,
  Whisper-transcribes your recording, and requires ≥ 0.90 similarity —
  proving the speaker read *this* script *now*. Registration IDs are
  single-use and expire after 10 minutes.
- **Synthesis responses are binary, not JSON.** Non-streaming TTS returns
  raw audio bytes (`audio/mpeg` by default); save with `-o`. Set
  `output_format` to `wav` (or any ffmpeg format) to change the container.
- **`stream=true` switches to SSE**: events `{"type":"speech.audio.delta","delta":"<base64 PCM>"}`
  (16-bit little-endian, 24 kHz mono) arrive as synthesis progresses; the
  HTTP stream simply ends at natural completion. A `speech.audio.done` event
  is the **failure terminator** — when you see it, synthesis failed (check
  service logs); it is not sent on success.
- **Cold-start latency is real.** The first synthesis after boot compiles
  the MLX graph and can exceed the internal 120 s synthesis deadline
  (surfacing as `504 tts_synthesis_failed`); a retry after warmup completes
  in ~20–45 s. Plan client timeouts accordingly.
- **Optional steering:** `style_prompt` (`{"kind":"preset","preset":"cheerful"}`
  or `{"kind":"custom","text":"…"}`), profanity scrub (`clean` + `locale`),
  and cross-lingual translation via Linga (`target`). `speed` is reserved
  and ignored.

## Step 1 — Get a user JWT

```bash verify=auto
USER_TOKEN=$(curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"device_id\": \"g61-demo\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"g61-speaker\"}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])')
test -n "${USER_TOKEN}" && echo "JWT acquired (${#USER_TOKEN} chars)"
```

**Expected** — a 24 h PS512 JWT. (`qa/login` requires `device_id` of at
least 8 chars; it provisions the user on first use via the Users services.)

## Step 2 — Record a voice sample and register (unverified flow)

Fabricate a clean test recording (macOS `say`, or use any microphone WAV —
the service conforms it to 24 kHz mono and rejects audio Whisper can't
transcribe with `422`):

```bash verify=auto
export WORKDIR=$(mktemp -d /tmp/g61-voice.XXXXXX)
say -o "${WORKDIR}/voice.wav" --data-format=LEI16@24000 \
  "Hello, this is my voice reference. I am reading this sentence clearly so the system can learn how I sound."
test -s "${WORKDIR}/voice.wav" && echo "sample recorded: $(stat -f%z ${WORKDIR}/voice.wav) bytes"

VP_JSON=$(curl -s -X POST "${SPEECH_URL}/v3/voice-prints" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}" \
  -F "file=@${WORKDIR}/voice.wav")
echo "${VP_JSON}"
VOICEPRINT_PATH=$(echo "${VP_JSON}" | python3 -c 'import json,sys; print(json.load(sys.stdin)["voice_print_wav_path"])')
echo "${VOICEPRINT_PATH}" | grep -q "speech-service/voice-prints/" && echo "voice print registered: ${VOICEPRINT_PATH}"
```

**Expected response** — `200`:
`{"voice_print_wav_path":"speech-service/voice-prints/<sha256(user sub)>/voice_print.wav"}`.
The multipart field must be exactly `file`; extra fields → `400`. This call
takes ~10 s (Whisper intelligibility check + CDN upload).

## Step 3 — Speak with the cloned voice

```bash verify=auto
curl -s -o "${WORKDIR}/spoken.mp3" -w 'HTTP %{http_code} | %{content_type} | %{size_download} bytes\n' \
  -X POST "${SPEECH_URL}/v3/text-to-speech" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}" \
  -d "{\"input\": \"Hello! This sentence is spoken with my cloned voice.\", \"reference_voiceprint_path\": \"${VOICEPRINT_PATH}\", \"locale\": \"en\"}"
file "${WORKDIR}/spoken.mp3"
ffprobe -v error -show_entries format=format_name,duration -of csv=p=0 "${WORKDIR}/spoken.mp3"
file "${WORKDIR}/spoken.mp3" | grep -qi "Audio file" && echo "synthesis verified"
```

**Expected** — `200` with an MP3 body (ID3-tagged MPEG audio, ~24 kHz
mono). `locale` (`en`, `ja-JP`, …) must match the language of `input`.
Change the container with `output_format: "wav"`. This is the slow call:
~20–45 s warm; first-call-after-boot can exceed the internal deadline and
return `504 tts_synthesis_failed` — retry rather than treat as fatal.

## Step 4 — Stream the audio as SSE

For play-as-it-generates clients, set `stream=true` and consume the SSE
events; each `delta` is base64 16-bit-LE PCM at 24 kHz mono:

```bash verify=auto
SSE_OUT=$(mktemp /tmp/g61-tts.XXXXXX.txt)
curl -s -N -X POST "${SPEECH_URL}/v3/text-to-speech" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}" \
  -d "{\"input\": \"Streaming check with the cloned voice.\", \"reference_voiceprint_path\": \"${VOICEPRINT_PATH}\", \"locale\": \"en\", \"stream\": true}" \
  > "${SSE_OUT}"
DELTAS=$(grep -c 'speech.audio.delta' "${SSE_OUT}")
echo "delta events: ${DELTAS}"
[ "${DELTAS}" -gt 0 ] || { echo "no audio deltas streamed"; exit 1; }
python3 - "${SSE_OUT}" <<'PY'
import base64, sys
pcm = b""
for line in open(sys.argv[1]):
    if line.startswith("data:") and "speech.audio.delta" in line:
        import json
        pcm += base64.b64decode(json.loads(line[5:])["delta"])
assert len(pcm) > 8000, len(pcm)  # > ~0.17 s of 16-bit 24 kHz mono
open(sys.argv[1] + ".pcm", "wb").write(pcm)
print(f"decoded PCM: {len(pcm)} bytes")
PY
```

**Expected** — several `data: {"type":"speech.audio.delta","delta":"…"}`
events (~10 s total); the HTTP response ends at natural completion (no
terminal event on success). A `speech.audio.done` event means synthesis
failed mid-stream.

## Step 5 — Verified enrollment (consent script)

For trusted enrollment, use the two-step verified flow. The service
generates a fresh script via Evergreen AI; you have 10 minutes to record it:

```bash verify=auto
BEGIN_JSON=$(curl -s -X POST "${SPEECH_URL}/v3/verified-voice-prints/verifiable-paragraphs" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}" \
  -d '{"locale": "en-US"}')
echo "${BEGIN_JSON}"
REG_ID=$(echo "${BEGIN_JSON}" | python3 -c 'import json,sys; print(json.load(sys.stdin)["registration_id"])')
SCRIPT=$(echo "${BEGIN_JSON}" | python3 -c 'import json,sys; print(json.load(sys.stdin)["script"])')
test -n "${REG_ID}" && echo "registration id: ${REG_ID} (TTL 600 s)"
```

**Expected response** — `200` with `registration_id` and a short
LLM-generated `script`. `503` here means Evergreen AI is unreachable.

Record the script (a human reads it; `say` stands in for CI), then
finalize:

```bash verify=auto
say -o "${WORKDIR}/verified.wav" --data-format=LEI16@24000 "${SCRIPT}"
VERIFIED_JSON=$(curl -s -X POST "${SPEECH_URL}/v3/verified-voice-prints/${REG_ID}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}" \
  -F "file=@${WORKDIR}/verified.wav")
echo "${VERIFIED_JSON}"
echo "${VERIFIED_JSON}" | python3 -c '
import json, sys
r = json.load(sys.stdin)
assert r["voice_print_wav_path"].startswith("speech-service/voice-prints/"), r
print("verified voice print:", r["voice_print_wav_path"])
'
```

**Expected** — `200` with the same `voice_print_wav_path` shape (the
verified print overwrites the unverified one — one print per user).
Failures: `404` (unknown/expired registration id), `422` (recording not
intelligible or transcript similarity < 0.90 vs. the script).

## Step 6 — Error contract

```bash verify=auto
# 400 — unknown voice print path on synthesis (plain text body)
CODE=$(curl -s -o /tmp/g61-err1.txt -w '%{http_code}' -X POST "${SPEECH_URL}/v3/text-to-speech" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}" \
  -d '{"input": "test", "reference_voiceprint_path": "speech-service/voice-prints/nobody/voice_print.wav", "locale": "en"}')
[ "${CODE}" = "400" ] && grep -q "Voice print not found" /tmp/g61-err1.txt \
  && echo "400 plain-text error ok: $(cat /tmp/g61-err1.txt)"

# 401 — missing JWT on registration (JSON body)
CODE=$(curl -s -o /tmp/g61-err2.txt -w '%{http_code}' -X POST "${SPEECH_URL}/v3/voice-prints" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -F "file=@${WORKDIR}/voice.wav")
[ "${CODE}" = "401" ] && echo "401 auth error ok: $(cat /tmp/g61-err2.txt)"
```

**Expected** — `400` with a plain-text body for a missing voice print, and
`401` JSON when the JWT is absent. (Some validation failures elsewhere in
this service return RFC-7807 `Problem` JSON — parse by status first, then
sniff the body.)

## Putting it together

```bash
KEYS=(-H "x-api-key: $INAYA_API_KEY" -H "x-client-app-key: $INAYA_APP_KEY")

# 1. Register (unverified)
VP=$(curl -sf -X POST "$SPEECH_URL/v3/voice-prints" -H "Authorization: Bearer $USER_TOKEN" \
  "${KEYS[@]}" -F "file=@voice.wav" | python3 -c 'import json,sys; print(json.load(sys.stdin)["voice_print_wav_path"])')

# 2. Speak (binary MP3 out)
curl -sf -o spoken.mp3 -X POST "$SPEECH_URL/v3/text-to-speech" \
  -H "Authorization: Bearer $USER_TOKEN" "${KEYS[@]}" -H "Content-Type: application/json" \
  -d "{\"input\": \"Status report ready.\", \"reference_voiceprint_path\": \"$VP\", \"locale\": \"en\"}"

# 3. Or stream (SSE base64 PCM deltas)
curl -sN -X POST "$SPEECH_URL/v3/text-to-speech" \
  -H "Authorization: Bearer $USER_TOKEN" "${KEYS[@]}" -H "Content-Type: application/json" \
  -d "{\"input\": \"Status report ready.\", \"reference_voiceprint_path\": \"$VP\", \"locale\": \"en\", \"stream\": true}"
```

(`SPEECH_URL` is `http://127.0.0.1:40161` on the local stack; `AUTH_URL`
is `http://127.0.0.1:40004`; `TENANT_ID` is your tenant. Voice-print
registration also needs the **CDN** on `:40040`; the verified flow adds
**Memcache** on `:40011` and **Evergreen AI** on `:40071`.)

## Try it live

Open the [Playground](../../playgrounds/playground.html#translation) →
**Translation & Voice Cloning** — the *Speech* console (VoicePrint +
Speak modes) exercises the same register → synthesize cycle against the
live service.

## Copy agent prompt

```prompt
You are working with the Inaya Labs Speech service at SPEECH_URL (:40161,
prefix /v3), which runs a LOCAL Qwen3-TTS voice-cloning model (no cloud
TTS key). The tenant key pair is in INAYA_API_KEY / INAYA_APP_KEY and
TENANT_ID; AUTH_URL (:40004) mints user JWTs via POST /v3/auth/qa/login
{"device_id" (min 8 chars), "locale": "en-US", "tenant_id", "username"}.
EVERY call needs the key-pair headers AND a user JWT (forwarded to the CDN
for voice-print download). Steps: (1) Fabricate a test voice recording with
macOS `say -o voice.wav --data-format=LEI16@24000 "<one clear sentence>".
(2) Register an UNVERIFIED voice print: POST SPEECH /v3/voice-prints with
multipart field `file=@voice.wav` — assert 200 and capture
voice_print_wav_path (a CDN path like speech-service/voice-prints/<hash>/
voice_print.wav). One print per user; re-registering overwrites. (3)
Synthesize: POST SPEECH /v3/text-to-speech {"input", "reference_voiceprint_path",
"locale": "en"} — the RESPONSE BODY IS BINARY MP3 (save with -o; ~20-45 s
warm; first-call-after-boot may 504 tts_synthesis_failed — retry). Assert
with `file` that it is MPEG audio. (4) Stream: repeat with "stream": true —
parse SSE `data: {"type":"speech.audio.delta","delta":"<base64 PCM>"}`
events (16-bit LE, 24 kHz mono), base64-decode and assert > 8 KB of PCM;
stream ends silently at completion (speech.audio.done is the FAILURE
terminator). (5) VERIFIED enrollment: POST /v3/verified-voice-prints/
verifiable-paragraphs {"locale":"en-US"} → capture registration_id (TTL
600 s) and script; record the script and POST it to /v3/verified-voice-
prints/{registration_id} with -F file=@recording.wav — assert 200 (422 =
transcript similarity < 0.90). (6) Errors: unknown voiceprint path on TTS
→ 400 PLAIN TEXT "Voice print not found"; missing JWT on registration →
401 JSON. Report status codes, byte counts, and PCM sizes per step; never
log the JWT.
```

## API reference

- [Speech Service — Swagger UI](https://speech-service-sandbox.inayalabs.com/v3/swagger-ui/)

## Next journeys

- **Translating text with Linga** (G-60) — the translation engine behind cross-lingual speech.
- **Transforming audio, images & video** (G-50) — trim/transcode the audio you just produced.
- **Uploading files** (G-02) — how files reach the CDN that stores voice prints.

### G-62 · Verifying audio provenance
URL: /docs/guides/verifying-audio-provenance.html

> Check whether an audio clip was produced by this platform — the Speech service watermarks every synthesized clip invisibly, and one multipart call returns a verdict, a confidence score, and (when the payload survives) the voice-print hint and request ID that created it.

> **Journey.** As a developer, I need to know whether an audio clip —
> submitted by a user, scraped from social media, or received through a
> moderation queue — was generated by *my* deployment, and ideally *which
> request* produced it, without exposing anyone's biometric data.

---

Every clip synthesized by the **Speech service** (`:40161`, prefix `/v3`)
carries an **invisible dual watermark**, embedded automatically before
encoding — there is nothing to enable per-request:

1. A **robust detection gate** (AudioSeal) answers *"was this audio
   produced here?"*. It survives MP3/AAC distribution, resampling,
   loudness normalization, and quiet recordings down to −20 dB, because
   detection peak-normalizes the audio before scoring.
2. A **128-bit payload** (WavMark) answers *"who and which request"*: a
   privacy-safe 4-byte HMAC **hint** of the voice print — the raw voice
   print UUID is never exposed — plus the low 64 bits of the request ID,
   CRC-verified.

One multipart `POST` to `/v3/audio/provenance` returns both layers.

## Prerequisites

- The Speech service with watermarking enabled: `WATERMARK_ENABLED=true`
  and the four ONNX model paths resolvable (defaults live under
  `services-reusable/speech-service/assets/models/`; regenerate them with
  `speech-service/wm-exporter`). When the engine is off, the check returns
  `503` — an explicit, honest answer rather than a fake negative.
- Any audio file: WAV, MP3, AAC, … — the service conforms it to 24 kHz
  mono before detection. `ffmpeg` on `PATH` is required by the service.
- Auth is the standard pair: `x-api-key` + `x-client-app-key` **and** a
  user JWT (Step 1). In sandbox/lower environments any or no JWT passes.

## Concepts

- **The response is three fields plus confidence.** `produced_by_service`
  (boolean gate verdict), `voice_print_hint` and `request_id_lo` (payload
  layer — both `null` when the 128-bit payload didn't survive the journey,
  even if the gate fired), and `confidence` in `[0, 1]`, computed as
  `0.4 * gate_score + 0.6 * payload_bit_accuracy` (so gate-only hits land
  in `[0.4, 1)`; clean audio scores near 0).
- **Gate and payload have different resilience.** The gate survives heavy
  processing (MP3 ≥ 96 kbps, telephone band-pass, −20 dB quiet recordings,
  EBU R128 normalization); the payload is pickier — expect it intact on
  lossless and light MP3, dropped on aggressive codecs. A
  `produced_by_service: true` with `null` payload fields is a *verified
  but untraceable* clip.
- **What defeats the check, by design:** tempo/speed shifts (≥ 3%),
  clipping that squashes the waveform body, and cropping away most of the
  clip — severe nonlinear damage is not invertible, and the service says
  `false` rather than guess.
- **The MCP tool `check_audio_provenance`** exposes the same check to
  agents (`/agent/v1/mcp/speech/*` SSE endpoint).
- **Privacy.** The voice-print hint is an HMAC-truncated 4-byte digest; it
  lets your back office correlate clips with voice prints, but cannot be
  reversed into the voice print or the user.

## Step 1 — Get a user JWT

```bash verify=auto
USER_TOKEN=$(curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"device_id\": \"g62-demo\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"g62-checker\"}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])')
test -n "${USER_TOKEN}" && echo "JWT acquired (${#USER_TOKEN} chars)"
```

**Expected** — a 24 h PS512 JWT (`qa/login` provisions the user on first
use; `device_id` must be ≥ 8 chars).

## Step 2 — Produce a watermarked clip

Any synthesis is automatically watermarked. If you don't have a voice
print yet, register one first — the full enrollment walk-through is
[Cloning a voice and speaking](cloning-a-voice-and-speaking.html); the
short version:

```bash verify=auto
export WORKDIR=$(mktemp -d /tmp/g62-prov.XXXXXX)
say -o "${WORKDIR}/voice.wav" --data-format=LEI16@24000 \
  "Hello, this is my voice reference for the provenance check guide."
VP_PATH=$(curl -s -X POST "${SPEECH_URL}/v3/voice-prints" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}" \
  -F "file=@${WORKDIR}/voice.wav" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["voice_print_wav_path"])')
echo "voice print: ${VP_PATH}"

curl -s -o "${WORKDIR}/spoken.mp3" -w 'HTTP %{http_code} | %{size_download} bytes\n' \
  -X POST "${SPEECH_URL}/v3/text-to-speech" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}" \
  -d "{\"input\": \"This sentence carries an invisible watermark.\", \"reference_voiceprint_path\": \"${VP_PATH}\", \"locale\": \"en\", \"output_format\": \"wav\"}"
file "${WORKDIR}/spoken.mp3" | grep -qi audio && echo "synthesized clip ready (watermarked at synthesis time)"
```

**Expected** — `200` with a WAV body. Synthesis is the slow call (~20–45 s
warm; a first-call-after-boot can hit the internal deadline and return
`504 tts_synthesis_failed` — retry). The watermark is embedded before
encoding — re-encoding the result to MP3 yourself does *not* remove it.

> **Re-speak output verifies identically.** `POST /v3/respeak`
> (multipart `payload` JSON + `file` audio) re-renders a recording in the
> registered voice — transcribe → optional enhancement → synthesize — and
> watermarks the result on the same path. Verified live: a re-spoken clip
> returns `produced_by_service: true` with `confidence: 1.0`, the same
> voice-print hint as TTS output for that voice print, and a fresh
> `request_id_lo` per request.

## Step 3 — Check the synthesized clip (positive verdict)

```bash verify=auto
curl -s -X POST "${SPEECH_URL}/v3/audio/provenance" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}" \
  -F "audio=@${WORKDIR}/spoken.mp3" \
  -o "${WORKDIR}/verdict.json"
python3 - "${WORKDIR}/verdict.json" <<'PY'
import json, sys
d = json.load(open(sys.argv[1]))
assert d["produced_by_service"] is True, d
assert d["voice_print_hint"], "payload fields should be present on a fresh WAV"
assert d["request_id_lo"] is not None, d
print(f'produced_by_service={d["produced_by_service"]} confidence={d["confidence"]:.4f} hint={d["voice_print_hint"]} request_id_lo={d["request_id_lo"]}')
PY
```

**Expected response** — `200`:
`{"produced_by_service":true,"request_id_lo":<u64>,"voice_print_hint":"<8 hex chars>","confidence":~0.99}`.
Detection takes a few seconds (decode + two ONNX model families). The
multipart field must be `audio` (or `file`); a missing field → `400`.

## Step 4 — Check audio the platform did not produce (negative verdict)

```bash verify=auto
say -o "${WORKDIR}/not-ours.wav" --data-format=LEI16@24000 \
  "This recording was never touched by the speech service."
curl -s -X POST "${SPEECH_URL}/v3/audio/provenance" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}" \
  -F "audio=@${WORKDIR}/not-ours.wav" \
  -o "${WORKDIR}/verdict-negative.json"
python3 - "${WORKDIR}/verdict-negative.json" <<'PY'
import json, sys
d = json.load(open(sys.argv[1]))
assert d["produced_by_service"] is False, d
assert d["voice_print_hint"] is None and d["request_id_lo"] is None, d
print(f'produced_by_service={d["produced_by_service"]} confidence={d["confidence"]:.4f}')
PY
```

**Expected response** — `200` with `produced_by_service:false`, payload
fields `null`, and `confidence` near 0 (typically < 0.05). Note the check
is a *positive* proof only: `false` means "no platform watermark detected",
not "this audio is authentic human speech".

## Step 5 — What survives a realistic distribution chain

Re-encode and re-check the same clip — the gate holds even where the
payload starts dropping:

```bash verify=auto
ffmpeg -y -loglevel error -i "${WORKDIR}/spoken.mp3" -codec:a libmp3lame -b:a 128k "${WORKDIR}/spoken-128k.mp3"
ffmpeg -y -loglevel error -i "${WORKDIR}/spoken.mp3" -af volume=-14dB "${WORKDIR}/spoken-quiet.wav"
for f in spoken-128k.mp3 spoken-quiet.wav; do
  echo -n "${f}: "
  curl -s -X POST "${SPEECH_URL}/v3/audio/provenance" \
    -H "x-api-key: ${INAYA_API_KEY}" \
    -H "x-client-app-key: ${INAYA_APP_KEY}" \
    -H "Authorization: Bearer ${USER_TOKEN}" \
    -F "audio=@${WORKDIR}/${f}" \
    -o "${WORKDIR}/verdict-${f}.json"
  python3 - "${WORKDIR}/verdict-${f}.json" <<'PY'
import json, sys
d = json.load(open(sys.argv[1]))
print(f'produced_by_service={d["produced_by_service"]} confidence={d["confidence"]:.4f} hint={d["voice_print_hint"]}')
PY
done
```

**Expected** — both stay `produced_by_service:true`: the 128 kbps MP3
typically keeps the full payload (hint present), while the −14 dB file
detects via the normalized gate — the quiet-recording scenario that
peak-normalization in the detect path exists for.

## Where to go next

- [Cloning a voice and speaking](cloning-a-voice-and-speaking.html) — the
  enrollment and synthesis journey this guide checks.
- [Transforming media](transforming-media.html) — re-encode, resample, and
  denoise chains for your media pipeline.
- API reference — Swagger UI at `/v3/swagger-ui/` on the Speech service,
  and the `check_audio_provenance` tool in the service's MCP surface.

### G-70 · Caching with Memcache
URL: /docs/guides/caching-with-memcache.html

> Add, read, expire, and delete tenant-isolated key-value entries on the shared Memcache service — exact-match caching with TTLs, a private loopback router for prefix listing and diagnostics, and MCP tools for agents.

> **Journey.** As a developer, I want a fast, shared, tenant-isolated cache
> that any of my services can use — set a value with a TTL, read it back,
> let it expire, and introspect what is cached — without running my own
> Redis cluster.

---

The Memcache service is the platform's shared key-value cache: one in-memory
store per tenant, AES-encrypted at rest, with exact-match lookups and
per-entry TTLs. Platform services use it to memoize expensive computations;
your services can use it the same way.

| Router | Port (local) | Audience | Routes |
|:---|:---|:---|:---|
| **Public** | `:40011` | Your services & agents | `add`, `get`, `delete` (+ MCP tools) |
| **Private (loopback)** | `:40001` | Ops & platform services only | prefix listing/deletion, per-tenant flush, diagnostics |

Both routers require the **tenant key pair and a user JWT** (the Swagger UI
and OpenAPI JSON are exempt). The private router exists on loopback precisely
so administrative operations — listing every entry under a key prefix,
flushing a tenant, reading hit-ratio counters — are never exposed to callers
outside the host.

## Prerequisites

- The tenant key pair (`x-api-key` / `x-client-app-key`) and a user JWT from
  `POST /v3/auth/qa/login`. Sandbox accepts any tenant id; the snippets use
  `playground_a43b973c77e94b7f8b0e64d96c86edf8`.

## Concepts

- **Exact match only.** Lookups compare keys literally — `weather:nyc` and
  `weather:sf` are unrelated entries. Semantic (similarity-based) caching is
  **not implemented** in this version of the service; plan your key names
  accordingly.
- **Two routers, one store.** Entries written on the public router are
  visible on the private router (and vice versa) — they share the same
  tenant cache. The split is about *who may call which operation*, not about
  separate data.
- **Values are encrypted at rest.** Entries are AES-encrypted in the shared
  store; the private router's prefix listing decrypts them for you.
- **TTL is required.** Every `add` carries `ttl_in_seconds`. A TTL of `1`
  expires after roughly a second; **a TTL of `0` is silently accepted but
  stores nothing** (see Step 3).
- **Plain-text bodies, not JSON.** `get` returns the raw value with
  `Content-Type: text/plain`; misses return an **empty `404`** (no JSON
  error body). `add` and `delete` likewise return empty bodies (`201` / `200`).
- **Capacity is per tenant.** Up to 10 million entries per tenant; on
  overflow the least-recently-used entry is evicted **silently** — a `get`
  you expected to hit can come back `404` under memory pressure.

## Step 1 — Add a value and read it back

Obtain a user JWT, then add an entry and read it back on the public router:

```bash verify=auto
USER_TOKEN=$(curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"device_id\": \"g70-demo\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"g70-cacher\"}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])')
echo "JWT obtained (${#USER_TOKEN} chars)"

ADD_CODE=$(curl -s -o /dev/null -w "%{http_code}" -X POST \
  "${MEMCACHE_URL}/v3/mem-cache/add/${TENANT_ID}" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}" \
  -d '{"key": "weather-cache-key", "value": "sunny 72F", "ttl_in_seconds": 300}')
echo "add status: ${ADD_CODE}"
test "${ADD_CODE}" = "201" && echo "entry stored"
```

**Expected** — `add status: 201` with an **empty body**. The key must be
8–120 characters and the value at most 2 MiB. `ttl_in_seconds` is mandatory.

Reading back returns the **raw value** as `text/plain` — not a JSON envelope:

```bash verify=auto
GET_RESP=$(curl -s -w "\n%{http_code}" \
  "${MEMCACHE_URL}/v3/mem-cache/get/${TENANT_ID}/weather-cache-key" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}")
echo "${GET_RESP}"
VALUE=$(echo "${GET_RESP}" | sed '$d')
CODE=$(echo "${GET_RESP}" | tail -n 1)
test "${CODE}" = "200" && test "${VALUE}" = "sunny 72F" && echo "round trip verified"
```

**Expected** — the body is exactly `sunny 72F` followed by a captured
`200` status line, then `round trip verified`. No JSON parsing needed —
though the MCP `get_cache_entry` tool does wrap it in JSON (see Step 4).

## Step 2 — Misses and deletes

A miss is a `404` with an **empty body** — distinguish it from an error by
the status code alone (and from a stored empty string by calling `add`
first). Delete returns `200` empty; afterwards the key misses:

```bash verify=auto
MISS_CODE=$(curl -s -o /dev/null -w "%{http_code}" \
  "${MEMCACHE_URL}/v3/mem-cache/get/${TENANT_ID}/never-written-key" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}")
echo "miss status: ${MISS_CODE} (empty body)"
test "${MISS_CODE}" = "404" && echo "miss semantics verified"

DEL_CODE=$(curl -s -o /dev/null -w "%{http_code}" -X DELETE \
  "${MEMCACHE_URL}/v3/mem-cache/delete/${TENANT_ID}/weather-cache-key" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}")
echo "delete status: ${DEL_CODE}"
test "${DEL_CODE}" = "200" || exit 1

GET_AFTER=$(curl -s -o /dev/null -w "%{http_code}" \
  "${MEMCACHE_URL}/v3/mem-cache/get/${TENANT_ID}/weather-cache-key" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}")
echo "get-after-delete status: ${GET_AFTER}"
test "${GET_AFTER}" = "404" && echo "delete verified"
```

**Expected** — `miss status: 404`, `delete status: 200`, then
`get-after-delete status: 404`.

## Step 3 — TTL expiry (and the `ttl=0` gotcha)

A short TTL expires quickly; give a 1-second entry two seconds and it is
gone. This is the mechanism behind "memoize for N seconds" patterns:

```bash verify=auto
curl -s -o /dev/null -w "add: %{http_code}\n" -X POST \
  "${MEMCACHE_URL}/v3/mem-cache/add/${TENANT_ID}" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}" \
  -d '{"key": "short-lived-key", "value": "ephemeral", "ttl_in_seconds": 1}'
sleep 2
CODE=$(curl -s -o /dev/null -w "%{http_code}" \
  "${MEMCACHE_URL}/v3/mem-cache/get/${TENANT_ID}/short-lived-key" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}")
echo "after 2s: ${CODE}"
test "${CODE}" = "404" && echo "TTL expiry verified"
```

**Expected** — `add: 201`, then `after 2s: 404` and `TTL expiry verified`.

One wart to know: **`ttl_in_seconds: 0` is accepted with `201` but stores
nothing** — a subsequent `get` misses. If you treat "zero" as "no expiry"
(as some cache APIs do), you will silently lose data. Always pass a real TTL:

```bash verify=auto
ZERO_CODE=$(curl -s -o /dev/null -w "%{http_code}" -X POST \
  "${MEMCACHE_URL}/v3/mem-cache/add/${TENANT_ID}" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}" \
  -d '{"key": "zero-ttl-key", "value": "lost forever", "ttl_in_seconds": 0}')
echo "ttl=0 add status: ${ZERO_CODE} (looks successful...)"
GET_ZERO=$(curl -s -o /dev/null -w "%{http_code}" \
  "${MEMCACHE_URL}/v3/mem-cache/get/${TENANT_ID}/zero-ttl-key" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}")
echo "ttl=0 get status: ${GET_ZERO} (...but nothing was stored)"
test "${ZERO_CODE}" = "201" && test "${GET_ZERO}" = "404" && echo "ttl=0 no-op documented"
```

**Expected** — `ttl=0 add status: 201`, then `ttl=0 get status: 404`:
a false-success gotcha. Use a generous TTL (or re-`add` on miss) instead.

## Step 4 — The private router: list by prefix, flush, and diagnostics

The loopback router (`:40001` locally) exposes administrative operations
that are deliberately absent from the public surface. All of them need the
same key pair **and JWT** — the router enforces authentication too.

Re-create two entries, then list everything under a key prefix (values come
back decrypted):

```bash verify=auto
for kv in "g70-alpha:one" "g70-beta:two"; do
  KEY=${kv%%:*}; VAL=${kv#*:}
  curl -s -o /dev/null -w "add ${KEY}: %{http_code}\n" -X POST \
    "${MEMCACHE_URL}/v3/mem-cache/add/${TENANT_ID}" \
    -H "Content-Type: application/json" \
    -H "x-api-key: ${INAYA_API_KEY}" \
    -H "x-client-app-key: ${INAYA_APP_KEY}" \
    -H "Authorization: Bearer ${USER_TOKEN}" \
    -d "{\"key\": \"${KEY}\", \"value\": \"${VAL}\", \"ttl_in_seconds\": 600}"
done

LIST=$(curl -s \
  "${MEMCACHE_INTERNAL_URL}/v3/mem-cache-internal/${TENANT_ID}/entries/prefix/g70" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}")
echo "${LIST}" | python3 -c '
import json, sys
entries = json.load(sys.stdin)
keys = sorted(e["key"] for e in entries)
assert keys == ["g70-alpha", "g70-beta"], keys
values = {e["key"]: e["value"] for e in entries}
assert values["g70-alpha"] == "one" and values["g70-beta"] == "two", values
print("prefix list verified:", keys)
'
```

**Expected** — a `200` JSON array `[{"key":"g70-alpha","shard":null,"value":"one"},…]`
(the `shard` field is currently always `null`) and `prefix list verified`.
The prefix match is literal string-prefix on keys.

Diagnostics report entry counts and hit/miss counters — useful for cache
effectiveness checks. The counters are **process-global** (across all tenants)
and reset to zero when the service restarts, so assert *deltas*, not absolute
values. Here we take a baseline, perform exactly one hit (the `g70-alpha`
entry from above) and one miss, and verify the counters moved accordingly:

```bash verify=auto
COUNT=$(curl -s "${MEMCACHE_INTERNAL_URL}/v3/mem-cache-internal/diagnostics/entry-count" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}")
echo "entry count: ${COUNT}"
echo "${COUNT}" | python3 -c '
import json, sys
n = json.load(sys.stdin)["number_of_entries"]
assert n >= 2, n  # at least the two g70-* entries from above
print("entry count verified:", n)
'

BASE=$(curl -s "${MEMCACHE_INTERNAL_URL}/v3/mem-cache-internal/diagnostics/performance-stats" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}")
echo "baseline: ${BASE}"

curl -s -o /dev/null "${MEMCACHE_URL}/v3/mem-cache/get/${TENANT_ID}/g70-alpha" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}"
curl -s -o /dev/null "${MEMCACHE_URL}/v3/mem-cache/get/${TENANT_ID}/g70-never-written" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}"

AFTER=$(curl -s "${MEMCACHE_INTERNAL_URL}/v3/mem-cache-internal/diagnostics/performance-stats" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}")
echo "after 1 hit + 1 miss: ${AFTER}"

BASE_HITS=$(echo "${BASE}" | python3 -c 'import json,sys; print(json.load(sys.stdin)["cache_hits"])')
BASE_MISSES=$(echo "${BASE}" | python3 -c 'import json,sys; print(json.load(sys.stdin)["cache_misses"])')
BASE_ACCESS=$(echo "${BASE}" | python3 -c 'import json,sys; print(json.load(sys.stdin)["cache_accesses"])')
echo "${AFTER}" | BASE_HITS="${BASE_HITS}" BASE_MISSES="${BASE_MISSES}" BASE_ACCESS="${BASE_ACCESS}" python3 -c '
import json, os, sys
stats = json.load(sys.stdin)
assert stats["cache_hits"] == int(os.environ["BASE_HITS"]) + 1, stats
assert stats["cache_misses"] == int(os.environ["BASE_MISSES"]) + 1, stats
assert stats["cache_accesses"] == int(os.environ["BASE_ACCESS"]) + 2, stats
assert abs(stats["cache_hit_ratio"] - 100.0 * stats["cache_hits"] / stats["cache_accesses"]) < 0.01, stats
print("performance stats verified: +1 hit, +1 miss, ratio is a 0-100 percentage")
'
```

**Expected** — `entry count verified: N` with N ≥ 2 (the two `g70-*` entries,
plus anything still live from Steps 1–3), then `after 1 hit + 1 miss` shows
each counter up by exactly one (two for `cache_accesses`) versus the
baseline, and `cache_hit_ratio` recomputed as a **0–100 percentage**
(e.g. `50.0` after an even mix). Only public-router `get` calls count toward
these counters; private-router listing and diagnostics reads do not.

Finally, wipe the prefix and confirm it is gone. (Related routes:
`DELETE /v3/mem-cache-internal/admin/{tenant_id}/entries` flushes one tenant,
`DELETE /v3/mem-cache-internal/admin/entries` flushes all — the multi-tenant
flush is for operators, not apps.)

```bash verify=auto
WIPE=$(curl -s -o /dev/null -w "%{http_code}" -X DELETE \
  "${MEMCACHE_INTERNAL_URL}/v3/mem-cache-internal/${TENANT_ID}/entries/prefix/g70" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}")
echo "prefix wipe status: ${WIPE}"
test "${WIPE}" = "200" || exit 1

REMAIN=$(curl -s \
  "${MEMCACHE_INTERNAL_URL}/v3/mem-cache-internal/${TENANT_ID}/entries/prefix/g70" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}")
echo "remaining under prefix: ${REMAIN}"
test "${REMAIN}" = "[]" && echo "prefix wipe verified"
```

**Expected** — `prefix wipe status: 200`, then `remaining under prefix: []`.

## Step 5 — Error contract and quirks

A few behaviors worth designing around — verified against the live service:

```bash verify=auto
# (a) Unauthenticated: 401 JSON, on both routers
UNAUTH=$(curl -s -w "\n%{http_code}" \
  "${MEMCACHE_URL}/v3/mem-cache/get/${TENANT_ID}/weather-cache-key" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}")
echo "${UNAUTH}"
echo "${UNAUTH}" | sed '$d' | grep -q '"Missing Authorization token"' \
  && test "$(echo "${UNAUTH}" | tail -n 1)" = "401" \
  && echo "unauthenticated 401 verified"

# (b) Short key: 400 plain text on add — but 500 on get (asymmetric wart)
SHORT_ADD=$(curl -s -w "\n%{http_code}" -X POST \
  "${MEMCACHE_URL}/v3/mem-cache/add/${TENANT_ID}" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}" \
  -d '{"key": "tiny", "value": "v", "ttl_in_seconds": 60}')
echo "${SHORT_ADD}"
SHORT_GET=$(curl -s -o /dev/null -w "%{http_code}" \
  "${MEMCACHE_URL}/v3/mem-cache/get/${TENANT_ID}/tiny" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}")
echo "short-key get status: ${SHORT_GET} (500, not 400 — known asymmetry)"
test "$(echo "${SHORT_ADD}" | tail -n 1)" = "400" && test "${SHORT_GET}" = "500" \
  && echo "short-key asymmetry documented"
```

**Expected** — (a) a `401` body `{"error":"Missing Authorization token"}`;
(b) add rejects the 4-character key with `400 Key is too small` (plain text),
while the same short key on `get` returns **500** — a known asymmetry:
validate key length client-side and treat any `5xx` from `get` as
"bad request shape", not "server down".

Two more quirks, stated without live demos:

- **`tags` and `shard` are accepted but discarded.** The `add` payload
  accepts these fields and the private listing always reports
  `"shard": null`; do not rely on them for grouping or routing.
- **Oversized values are rejected.** The value limit is 2 MiB per entry;
  per-tenant capacity is 10 M entries with silent LRU eviction.

And the one thing this version does **not** do: **semantic caching**. There
is no fuzzy/similarity lookup — `get` is exact-match only. (A semantic-match
library ships in the workspace but is not wired into this service version;
brochure claims to that effect are aspirational.)

## Putting it together

A compact cache-aside pattern: check the cache, compute on miss, store with
a TTL, and use the private router for introspection:

```bash
# cache-aside read for "expensive-computation"
KEY="expensive-computation"
HIT=$(curl -s -o /tmp/g70-hit.txt -w "%{http_code}" \
  "${MEMCACHE_URL}/v3/mem-cache/get/${TENANT_ID}/${KEY}" \
  -H "x-api-key: ${INAYA_API_KEY}" -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}")
if [ "${HIT}" = "200" ]; then
  RESULT=$(cat /tmp/g70-hit.txt); echo "cache hit: ${RESULT}"
else
  RESULT=$(python3 -c 'import hashlib; print(hashlib.sha256(b"work").hexdigest()[:16])')
  echo "computed: ${RESULT}; storing for 5 min"
  curl -s -o /dev/null -X POST "${MEMCACHE_URL}/v3/mem-cache/add/${TENANT_ID}" \
    -H "Content-Type: application/json" \
    -H "x-api-key: ${INAYA_API_KEY}" -H "x-client-app-key: ${INAYA_APP_KEY}" \
    -H "Authorization: Bearer ${USER_TOKEN}" \
    -d "{\"key\": \"${KEY}\", \"value\": \"${RESULT}\", \"ttl_in_seconds\": 300}"
fi

# ops introspection on the loopback router
curl -s "${MEMCACHE_INTERNAL_URL}/v3/mem-cache-internal/diagnostics/performance-stats" \
  -H "x-api-key: ${INAYA_API_KEY}" -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}"; echo
```

(`MEMCACHE_URL` / `MEMCACHE_INTERNAL_URL` are the public/loopback base URLs —
`http://127.0.0.1:40011` / `http://127.0.0.1:40001` locally; `AUTH_URL` is
`http://127.0.0.1:40004`; `TENANT_ID` is your tenant.)

## Try it live

Open the [Playground](../../playgrounds/playground.html#infra) →
**Infrastructure & Storage** — the *Memcache* console exercises add / get /
delete against the live service.

Agents can also call the cache directly: the service exposes MCP tools
`add_cache_entry`, `get_cache_entry`, and `delete_cache_entry` on the public
router (`/agent/v1/mcp`, streamable HTTP, spec 2026-07-28), under the same key
pair + JWT authentication.

## Copy agent prompt

```prompt
You are working with the Inaya Labs Memcache service. The public router is
MEMCACHE_URL (:40011), the private loopback router is MEMCACHE_INTERNAL_URL
(:40001), Auth is AUTH_URL (:40004); the tenant key pair is in INAYA_API_KEY /
INAYA_APP_KEY and TENANT_ID. EVERY route on BOTH routers requires the key pair
headers AND a user JWT: obtain it via POST AUTH /v3/auth/qa/login
{"device_id","locale":"en-US","tenant_id","username"} -> .token. Steps:
(1) Add: POST MEMCACHE /v3/mem-cache/add/{tenant} {"key","value",
"ttl_in_seconds"} -> 201 EMPTY body. Key must be 8-120 chars; value <= 2 MiB;
ttl_in_seconds is REQUIRED and ttl=0 stores NOTHING (201 but immediate miss).
(2) Get: GET /v3/mem-cache/get/{tenant}/{key} -> 200 with the RAW value as
text/plain (not JSON); miss/expired -> 404 with EMPTY body. (3) Misses &
delete: DELETE /v3/mem-cache/delete/{tenant}/{key} -> 200 empty; a subsequent
get is 404. (4) TTL: ttl_in_seconds 1 expires after ~1-2 s (verify with sleep 2
then expect 404). (5) Private router (same auth!): GET
MEMCACHE_INTERNAL /v3/mem-cache-internal/{tenant}/entries/prefix/{prefix} ->
200 JSON array of {"key","shard":null,"value"} (decrypted); DELETE same path
wipes the prefix -> then list returns []; diagnostics GET
/v3/mem-cache-internal/diagnostics/entry-count -> {"number_of_entries":N} and
/v3/mem-cache-internal/diagnostics/performance-stats ->
{"cache_accesses","cache_hit_ratio","cache_hits","cache_misses"}. (6) Error
contract: no JWT -> 401 {"error":"Missing Authorization token"}; a short (<8
char) key -> 400 plain text on add but 500 on get (known asymmetry — validate
keys client-side); "tags"/"shard" fields are accepted on add but discarded;
lookups are EXACT MATCH ONLY — semantic caching is NOT implemented in this
version. Report status codes and bodies per step; never log the JWT.
```

## API reference

- [Memcache — Swagger UI](https://memcache-service-sandbox.inayalabs.com/v3/mem-cache/swagger-ui/) · [Memcache — OpenAPI JSON](https://memcache-service-sandbox.inayalabs.com/v3/mem-cache/api-docs/openapi.json)

## Next journeys

- **Storing and serving files with CDN** (G-71) — when your values are files, not strings.
- **Uploading files** (G-02) — the upload flow that places files into the CDN.
- **Errors, retries & idempotency** (G-04) — how to treat `404` misses and `5xx` responses in retry logic.

### G-71 · Storing & serving files with the CDN
URL: /docs/guides/storing-serving-files-cdn.html

> The CDN Service as tenant-scoped storage infrastructure — per-user ownership with 403 enforcement, time-limited download tokens (and what they do and don't exempt), delete lifecycle semantics, path-safety validation, limits and quotas, and MCP tools for agents.

> **Journey.** As a developer, I want to store files in a shared,
> tenant-scoped object store where every file has an owner, only the owner
> (or an admin) can read and delete it, and I can hand out time-limited
> download tokens — and I want to know exactly which failure code each
> mistake produces.

---

The CDN Service is the platform's file storage layer. Where [G-04
"Uploading files the platform way"](../guides/file-upload.html) walks the
upload journey end to end, this guide covers what makes the CDN **storage
infrastructure**: the ownership model, token lifecycle, delete semantics,
path safety, and limits — the parts you design around when several services
and users share one tenant.

| Aspect | Behavior |
|:---|:---|
| **Addressing** | Relative paths inside the caller's tenant namespace — there is no global bucket listing |
| **Ownership** | Every file records its owner's `sub` (from the JWT); reads/deletes/token-mints enforce owner-or-admin |
| **Access** | `private/download` for the owner; `download?token=…` scoped by a signed, expiring token (caller credentials still required) |
| **Payload limit** | 50 MiB per upload (`INAYA_LABS_CDN_MAX_PAYLOAD_BYTES`) |
| **Tenant quota** | 5 GiB per tenant (enforced at write time) |
| **Storage backends** | `inayalabs` today; the `storage_provider` parameter is reserved for future backends |
| **Token TTL** | 180 s default (`INAYA_LABS_CDN_TOKEN_TTL_IN_SECONDS`) |

## Prerequisites

- The tenant key pair (`x-api-key` / `x-client-app-key`) and **two** user
  JWTs from `POST /v3/auth/qa/login` — one per "user" — because this guide
  demonstrates cross-user access control. Sandbox accepts any tenant id; the
  snippets use `playground_a43b973c77e94b7f8b0e64d96c86edf8`.

## Concepts

- **Tenant namespace, per-user ownership.** Paths live under the caller's
  tenant, but the *owning principal* is the JWT `sub`. Two users in one
  tenant cannot see each other's files: a non-owner's read or delete returns
  `403`, even though the path exists.
- **"No ownership record" reads as 403, not 404.** Asking for a path that
  was never uploaded — or was deleted — returns `403 "You are not allowed to
  download this file"`, identical to being a non-owner. Existence is never
  revealed; design UIs accordingly.
- **Tokens scope, they don't anonymize.** A download token encodes the path,
  provider, expiry, and an HMAC signature, and it is bound to the owning
  tenant/user. But `GET /v3/cdn/files/download?token=…` **still requires the
  caller's key pair and JWT** — the token is a capability on top of caller
  identity, not a replacement for it.
- **Delete removes bytes and ownership together.** After `DELETE`, reads
  return the same 403 as for a never-existing path, and a second delete is
  also 403 (there is no ownership record left to authorize against).

## Step 1 — Round trip in a fresh namespace

Owner "A" uploads a file and reads it back. The multipart contract (`path` +
`file` fields) is covered in depth in [G-04](../guides/file-upload.html);
here we use it to set up the ownership fixtures the rest of the guide
exercises:

```bash verify=auto
TOKEN_A=$(curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"device_id\": \"g71-owner-01\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"g71-owner\"}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])')
echo "JWT A obtained (${#TOKEN_A} chars)"

printf 'quarterly report draft\n' > /tmp/g71-report.txt
UP=$(curl -s -o /dev/null -w "%{http_code}" -X POST \
  "${CDN_URL}/v3/cdn/files/upload" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${TOKEN_A}" \
  -F "path=g71/report.txt" -F "file=@/tmp/g71-report.txt")
echo "upload: ${UP}"
test "${UP}" = "201" || exit 1

BODY=$(curl -s "${CDN_URL}/v3/cdn/files/private/download?path=g71/report.txt" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${TOKEN_A}")
echo "downloaded: ${BODY}"
test "${BODY}" = "quarterly report draft" && echo "round trip verified"
```

**Expected** — `upload: 201` (empty body), then `downloaded: quarterly report
draft` and `round trip verified`. `GET /v3/cdn/files/private/download` takes
the path as a **query parameter** and returns the raw bytes with
`Content-Disposition`/`Content-Type` headers set.

## Step 2 — Ownership: a second user in the same tenant is a stranger

User "B" shares the tenant but has a different `sub`. B cannot read or
delete A's file — the error is indistinguishable from "file doesn't exist":

```bash verify=auto
TOKEN_B=$(curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"device_id\": \"g71-intruder-02\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"g71-intruder\"}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])')
echo "JWT B obtained (${#TOKEN_B} chars)"

DL_B=$(curl -s -w "\n%{http_code}" \
  "${CDN_URL}/v3/cdn/files/private/download?path=g71/report.txt" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${TOKEN_B}")
echo "${DL_B}"
DEL_B=$(curl -s -o /dev/null -w "%{http_code}" -X DELETE \
  "${CDN_URL}/v3/cdn/files/g71/report.txt" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${TOKEN_B}")
echo "B delete: ${DEL_B}"
test "$(echo "${DL_B}" | sed '$d')" = "You are not allowed to download this file" \
  && test "$(echo "${DL_B}" | tail -n 1)" = "403" && test "${DEL_B}" = "403" \
  && echo "ownership enforcement verified"
```

**Expected** — B's download returns `403` with the plain-text body
`You are not allowed to download this file`, and B's delete is `403` too.

**Admin bypass.** Callers whose JWT `roles` claim contains `Admin` or
`SuperAdmin` pass the ownership check for any file in the tenant. The QA
login issues `roles: "Subscriber"`, so the bypass can't be demonstrated with
the login shortcut — it is enforced in the ownership middleware and is the
escape hatch for support workflows.

## Step 3 — Download tokens: mint, verify, tamper, strip credentials

Owner A mints a token for the report and inspects it. The token is
base64-encoded and decodes to a signed record — expiry, provider, tenant,
owner, and path, separated by `:`, with an HMAC:

```bash verify=auto
TOK_RESP=$(curl -s -X POST "${CDN_URL}/v3/cdn/files/token" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${TOKEN_A}" \
  -d '{"path": "g71/report.txt"}')
echo "${TOK_RESP}"
DL_TOKEN=$(echo "${TOK_RESP}" | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])')

EXPIRES=$(echo "${TOK_RESP}" | python3 -c 'import json,sys; print(json.load(sys.stdin)["expires_at"])')
NOW=$(date +%s)
TTL=$((EXPIRES - NOW))
echo "token TTL: ${TTL}s (default 180s)"
test "${TTL}" -ge 170 && test "${TTL}" -le 180 || exit 1

DECODED=$(python3 -c "import base64,sys; t=sys.argv[1]; print(base64.b64decode(t + '=' * (-len(t) % 4)).decode())" "${DL_TOKEN}")
echo "decoded token record: ${DECODED}"
echo "${DECODED}" | grep -q "^${EXPIRES}:inayalabs:" && echo "token structure verified"
```

**Expected** — `{"token":"…","expires_at":<now+180>}`; the TTL check passes;
the decoded record looks like
`<expires_at>:inayalabs:<tenant>/<sub>/g71/report.txt:<hex-hmac>`.

The download works **with the caller's credentials** (as documented in
[G-04](../guides/file-upload.html)) — and fails without them. A flipped
character invalidates the HMAC:

```bash verify=auto
WITH_AUTH=$(curl -s -o /dev/null -w "%{http_code}" \
  "${CDN_URL}/v3/cdn/files/download?token=${DL_TOKEN}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${TOKEN_A}")
echo "download with credentials: ${WITH_AUTH}"
test "${WITH_AUTH}" = "200" || exit 1

NO_AUTH=$(curl -s -o /dev/null -w "%{http_code}" \
  "${CDN_URL}/v3/cdn/files/download?token=${DL_TOKEN}")
echo "download without credentials: ${NO_AUTH} (token does NOT replace auth)"

LAST_CHAR="${DL_TOKEN: -1}"
FLIPPED=$([ "${LAST_CHAR}" = "A" ] && echo B || echo A)
TAMPERED="${DL_TOKEN%?}${FLIPPED}"
TAMPER_RESP=$(curl -s -w "\n%{http_code}" \
  "${CDN_URL}/v3/cdn/files/download?token=${TAMPERED}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${TOKEN_A}")
echo "${TAMPER_RESP}"
test "${NO_AUTH}" = "401" \
  && test "$(echo "${TAMPER_RESP}" | sed '$d')" = "Invalid or expired token" \
  && test "$(echo "${TAMPER_RESP}" | tail -n 1)" = "401" \
  && echo "token verification verified"
```

**Expected** — `download with credentials: 200`; `download without
credentials: 401` (the token does not exempt a caller from the key pair +
JWT); the tampered token returns `401 Invalid or expired token`.

Two token quirks, stated plainly:

- **Missing token** returns `400 Failed to deserialize query string: missing
  field \`token\`` — Axum's query-deserialization rejection fires before the
  handler's own `400 "Query parameter token is required"` branch, so that
  friendlier message is currently unreachable.
- **Expired tokens** behave identically to tampered ones (`401 Invalid or
  expired token`). With the default 180 s TTL, waiting one TTL out is the
  only way to observe expiry — plan for short TTLs when testing revocation.

## Step 4 — Delete lifecycle: bytes and ownership go together

```bash verify=auto
DEL=$(curl -s -o /dev/null -w "%{http_code}" -X DELETE \
  "${CDN_URL}/v3/cdn/files/g71/report.txt" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${TOKEN_A}")
echo "owner delete: ${DEL}"
test "${DEL}" = "204" || exit 1

READ_AFTER=$(curl -s -w "\n%{http_code}" \
  "${CDN_URL}/v3/cdn/files/private/download?path=g71/report.txt" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${TOKEN_A}")
echo "${READ_AFTER}"
DEL_AGAIN=$(curl -s -o /dev/null -w "%{http_code}" -X DELETE \
  "${CDN_URL}/v3/cdn/files/g71/report.txt" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${TOKEN_A}")
echo "second delete: ${DEL_AGAIN}"
test "$(echo "${READ_AFTER}" | sed '$d')" = "You are not allowed to download this file" \
  && test "$(echo "${READ_AFTER}" | tail -n 1)" = "403" && test "${DEL_AGAIN}" = "403" \
  && echo "delete lifecycle verified"
```

**Expected** — owner delete: `204` (empty body); the owner's *own* subsequent
read returns `403 "You are not allowed to download this file"` (the ownership
record is gone — identical to the never-existed case); a second delete is
also `403`. **There is no idempotent "already gone" signal**: if your cleanup
logic needs to distinguish "deleted now" from "was already gone", track that
in your own state.

## Step 5 — Path safety and limits

Every path is validated component-by-component: `..`, absolute paths, and
root prefixes are rejected before touching storage. Unknown storage backends
are rejected at the query layer:

```bash verify=auto
TRAV=$(curl -s -w "\n%{http_code}" -X POST \
  "${CDN_URL}/v3/cdn/files/upload" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${TOKEN_A}" \
  -F "path=../escape.txt" -F "file=@/tmp/g71-report.txt")
echo "${TRAV}"
ABS=$(curl -s -w "\n%{http_code}" -X POST \
  "${CDN_URL}/v3/cdn/files/upload" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${TOKEN_A}" \
  -F "path=/etc/escape.txt" -F "file=@/tmp/g71-report.txt")
echo "${ABS}"
PROV=$(curl -s -w "\n%{http_code}" -X POST \
  "${CDN_URL}/v3/cdn/files/upload?storage_provider=local" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${TOKEN_A}" \
  -F "path=g71/x.txt" -F "file=@/tmp/g71-report.txt")
echo "${PROV}"
test "$(echo "${TRAV}" | sed '$d')" = "Invalid path" && test "$(echo "${TRAV}" | tail -n 1)" = "400" \
  && test "$(echo "${ABS}" | tail -n 1)" = "400" && test "$(echo "${PROV}" | tail -n 1)" = "400" \
  && echo "path safety and provider validation verified"
```

**Expected** — traversal and absolute paths: `400 Invalid path` (plain text);
bogus provider: `400` with `unknown variant \`local\`, expected \`inayalabs\``.
Only `inayalabs` exists today — the parameter is a forward-compatible hook,
so sending it explicitly pins you to the default backend.

**Limits** (from the service configuration, not demoed per-request):

- Max upload body: **50 MiB** (`INAYA_LABS_CDN_MAX_PAYLOAD_BYTES`); the
  router's `DefaultBodyLimit` enforces it.
- Per-tenant quota: **5 GiB**, enforced at write time.
- Per-request lifetime is bounded by a 30 s tower timeout (`408`).

## Error contract cheat sheet

| Situation | Status | Body |
|:---|:---|:---|
| Missing JWT (any route) | 401 | `{"error":"Missing Authorization token"}` |
| Non-owner read | 403 | `You are not allowed to download this file` |
| Non-owner delete / delete of missing file | 403 | (empty) |
| Owner read of deleted/never-existing path | 403 | `You are not allowed to download this file` |
| Re-upload without `overwrite=true` | 409 | (see [G-04](../guides/file-upload.html)) |
| `..` or absolute path | 400 | `Invalid path` |
| Unknown `storage_provider` | 400 | `unknown variant …, expected \`inayalabs\`` |
| Tokenized download, missing `token` | 400 | `Failed to deserialize query string: missing field \`token\`` |
| Tokenized download, bad/tampered/expired token | 401 | `Invalid or expired token` |

Notable asymmetry: **`404` essentially never appears** on the file routes —
the ownership check fires before (or instead of) existence checks, so
"gone" and "not yours" both read as `403`.

## Agents: MCP tools

Agents can manage files directly: the service exposes three MCP tools on
`/agent/v1/mcp` (streamable HTTP, spec 2026-07-28), under the same key pair + JWT
authentication — `upload_file` (content base64-encoded), `delete_file`
(owner-or-admin enforced), and `generate_download_token` (owner-or-admin;
returns the token + `expires_at`). Every tool mirrors its REST counterpart
and applies the same ownership rules.

## Putting it together

A small "secure drop" script — upload as A, mint a token, hand both the
token *and* the reminder that credentials are still required to B, clean up:

```bash
# A: upload + mint
curl -sf -X POST "${CDN_URL}/v3/cdn/files/upload" \
  -H "x-api-key: ${INAYA_API_KEY}" -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${TOKEN_A}" \
  -F "path=g71/drop/report.txt" -F "file=@./report.txt"
TOK=$(curl -sf -X POST "${CDN_URL}/v3/cdn/files/token" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${TOKEN_A}" -d '{"path": "g71/drop/report.txt"}' \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])')

# B (or any caller WITH credentials): download via token
curl -sf "${CDN_URL}/v3/cdn/files/download?token=${TOK}" \
  -H "x-api-key: ${INAYA_API_KEY}" -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${TOKEN_B}" -o ./report-copy.txt

# A: revoke-by-delete (tokens die with the file)
curl -sf -o /dev/null -X DELETE "${CDN_URL}/v3/cdn/files/g71/drop/report.txt" \
  -H "x-api-key: ${INAYA_API_KEY}" -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${TOKEN_A}"
```

(`CDN_URL` is `http://127.0.0.1:40040` locally; `AUTH_URL` is
`http://127.0.0.1:40004`; `TENANT_ID` is your tenant.)

## Try it live

Open the [Playground](../../playgrounds/playground.html#infra) →
**Infrastructure & Storage** — the *CDN* console exercises the
upload/download/delete cycle against the live service with a file picker.

## Copy agent prompt

```prompt
You are working with the Inaya Labs CDN Service at CDN_URL (:40040); Auth is
AUTH_URL (:40004); the tenant key pair is in INAYA_API_KEY / INAYA_APP_KEY and
TENANT_ID. Every file route requires the key pair AND a user JWT from POST
AUTH /v3/auth/qa/login {"device_id","locale":"en-US","tenant_id","username"}.
Steps: (1) OWNER A: upload POST CDN /v3/cdn/files/upload (multipart fields
"path" and "file") -> 201 empty; read back GET
/v3/cdn/files/private/download?path=<p> with A's JWT -> 200 raw bytes. (2)
OWNERSHIP: obtain JWT B (different username); B's private/download and DELETE
/v3/cdn/files/<p> both -> 403 (B's download body is the plain text "You are
not allowed to download this file"; indistinguishable from a missing file).
(3) TOKENS: A mints POST /v3/cdn/files/token {"path":<p>} -> 200
{"token","expires_at"} (TTL default 180s from
INAYA_LABS_CDN_TOKEN_TTL_IN_SECONDS); the token is base64 of
"<expiry>:inayalabs:<tenant>/<sub>/<path>:<hmac>". GET
/v3/cdn/files/download?token=<t> works WITH caller credentials (200) but
returns 401 WITHOUT them — tokens scope, they do not replace auth; a
single-character flip -> 401 "Invalid or expired token"; missing token -> 400
"Failed to deserialize query string: missing field `token`". (4) DELETE:
owner A deletes -> 204; A's own subsequent read -> 403 (ownership record
gone); second delete -> 403 (no idempotent already-gone signal). (5) PATH
SAFETY: upload paths "../x" or "/etc/x" -> 400 "Invalid path"; only
storage_provider "inayalabs" exists (others -> 400). Limits: 50 MiB upload,
5 GiB per tenant, 30 s request timeout. Note 404 essentially never appears —
403 covers non-owner AND missing. Report status codes and bodies per step;
never log the JWTs.
```

## API reference

- [CDN Service — Swagger UI](https://cdn-service-sandbox.inayalabs.com/v3/cdn/swagger-ui/) · [Scalar](https://cdn-service-sandbox.inayalabs.com/v3/cdn/scalar)

## Next journeys

- **Uploading files the platform way** (G-04) — the multipart upload journey, overwrite conflicts, and multi-language upload snippets.
- **Caching with Memcache** (G-70) — cache small derived values, not files.
- **Errors, retries & idempotency** (G-03) — retry discipline for uploads; note the CDN's delete is *not* idempotent (403 on repeat).


## Tutorials

### T-01 · Build a support assistant
URL: /docs/tutorials/support-assistant.html

> **Scenario.** *Acme Homeware*'s three-person support team answers the same
> questions all day: *"How long do I have to return this?"*, *"When will my
> order ship?"*. You will build them an assistant that answers from the
> company's FAQ — not from a model's imagination — and pings the team on a
> live notification stream when a customer sounds like they need a human.

This tutorial **composes** four guides instead of repeating them: it assumes
you have worked through (or can consult) [G-01 Authentication](../guides/authentication-sessions.html), [G-11 Grounding documents](../guides/grounding-documents.html), [G-10 First AI chat session](../guides/first-ai-chat-session.html), and the SSE pattern in [G-50 Transforming media](../guides/transforming-media.html). Where those guides explain a contract in depth, this tutorial links instead of re-teaching.

**Time:** ~45 min · **Level:** intermediate

## Architecture

```text
 customer question
        │
        ▼
┌───────────────────────┐   1. JWT (QA login)    ┌────────────────┐
│  support_assistant.sh  │ ─────────────────────▶ │  Auth :40004    │
│  (this tutorial)       │                        └────────────────┘
└───┬───────────┬───────┘
    │ 2. ingest │ 3. grounded chat (use_rag)
    ▼           ▼
┌─────────┐  ┌──────────────────┐
│ RAG      │◀─│ Evergreen AI      │  4. escalation keyword detected
│ :40100   │  │ :40071            │ ──────────────┐
└─────────┘  └──────────────────┘               ▼
                                    ┌──────────────────────┐   5. SSE
                                    │ Notifications         │ ──▶ support
                                    │ publish :40014        │     dashboard
                                    │ subscribe :40013      │
                                    └──────────────────────┘
```

The data plane is two hops: the FAQ lives in a **RAG collection**, and an
**Evergreen AI session** is created *grounded on that collection* — every
chat message is answered against the FAQ. The alert plane is a **topic** on
the Notifications service: the script publishes `support-escalations` events
on the loopback port, and any dashboard holding an SSE subscription on the
public port receives them.

## Prerequisites

- The local stack running: `auth-service` (`:40004`), `rag-service`
  (`:40100`), `evergreen-ai-service` (`:40071`), `notifications-service`
  (`:40013` public / `:40014` loopback).
- The tenant key pair in `INAYA_API_KEY` / `INAYA_APP_KEY` and a tenant id.
  The snippets use `playground_a43b973c77e94b7f8b0e64d96c86edf8`.
- `jq`-free — the snippets parse JSON with `python3` only.
- Run all snippets **from the repository root** (`/Users/test/projects/Software/InayaLabs`) — the companion code is addressed by relative path.
- The AI model backend running: `ollama serve` with the `Qwen3_VL_4B_Q4`
  model pulled (the local stack expects Ollama on `:11434`; the model maps to
  the pulled `qwen3-vl:4b` tag). The first grounded chat call pays the
  model-load cost — subsequent calls are fast.

## Companion code

```text
InayaLabs-website/docs-src/tutorials/_code/support-assistant/
├── faq.txt                 ← the corpus you will ingest
├── support_assistant.sh   ← the assembled driver (all checkpoints)
└── README.md              ← how to run it standalone
```

The checkpoint blocks below are the script's stages, one at a time. Download
links: [faq.txt](_code/support-assistant/faq.txt) · [support_assistant.sh](_code/support-assistant/support_assistant.sh) · [README.md](_code/support-assistant/README.md)

## Checkpoint 1 — Authenticate as the support app

The app needs a user JWT (the QA shortcut stands in for the production
G-01 flows). One JWT is enough — it is the *support app's* identity; the
escalation notification is published on the app's behalf:

```bash verify=auto
TOKEN=$(curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"device_id\": \"acme-support-01\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"acme-support-bot\"}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])')
echo "support-app JWT obtained (${#TOKEN} chars)"
test "${#TOKEN}" -gt 100 && echo "checkpoint 1 passed: authenticated"
```

**You should see** — `checkpoint 1 passed: authenticated`.

## Checkpoint 2 — Ingest the FAQ into a RAG collection

Create a dedicated collection (`support_faq` — **never use hyphens in
collection ids**, see G-11) and ingest `faq.txt`. Then prove the corpus is
searchable before wiring it to chat:

```bash verify=auto
COLLECTION="support_faq"
AUTH=(-H "Authorization: Bearer ${TOKEN}"
      -H "x-api-key: ${INAYA_API_KEY}"
      -H "x-client-app-key: ${INAYA_APP_KEY}")
FAQ="InayaLabs-website/docs-src/tutorials/_code/support-assistant/faq.txt"

DOC_ID=$(curl -sf -X POST "${RAG_URL}/v3/ingest?collection_id=${COLLECTION}" \
  "${AUTH[@]}" -H "X-File-Name: faq.txt" -H "Content-Type: application/octet-stream" \
  --data-binary @"${FAQ}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["document_id"])')
echo "ingested FAQ: ${DOC_ID}"

HIT=$(curl -sf -X POST "${RAG_URL}/v3/search" "${AUTH[@]}" \
  -H "Content-Type: application/json" \
  -d "{\"collection_id\": \"${COLLECTION}\", \"question\": \"How long do I have to return an item for a full refund?\"}")
echo "${HIT}" | python3 -c '
import json, sys
d = json.load(sys.stdin)
top = d["chunks"][0]
assert "30 days" in top["content"] or "thirty days" in top["content"].lower(), top["content"][:200]
print("checkpoint 2 passed: top hit", round(top["score"], 3), "mentions the 30-day window")
'
```

**You should see** — `ingested FAQ: <id>`, then
`checkpoint 2 passed: top hit 0.xxx mentions the 30-day window`. If the top
hit is off-topic, the ingestion failed — re-run the block (ingest is
upsert, not duplicate).

> [!PITFALL]
> The RAG ingestor keys on the declared file name and rejects unknown
> extensions — `X-File-Name: faq.md` fails with `400 unsupported format
> 'faq.md'. Supported: .pdf, .docx, .pptx, .xlsx, .txt`. Markdown is not a
> supported ingest format; the companion corpus is therefore `faq.txt`.

## Checkpoint 3 — Answer a customer question, grounded

Create an Evergreen session **grounded on the collection** (`use_rag`), then
ask the exact question customers ask a hundred times a day. The session pins
the small `Qwen3_VL_4B_Q4` model so the tutorial is reproducible; production
would use capability criteria (G-10):

```bash verify=auto
SESSION=$(curl -s -X POST "${EVG_URL}/v3/sessions" \
  "${AUTH[@]}" -H "Content-Type: application/json" \
  -d '{
    "required_modalities": ["Text"],
    "selection_criteria": [ { "Model": { "optionality": "MustHave", "name": "Qwen3_VL_4B_Q4" } } ],
    "system_prompt": "You are the Acme Homeware support assistant. Answer only from the provided FAQ context, in one short sentence.",
    "temperature": 0.2,
    "use_rag": { "collection_id": "'"${COLLECTION}"'", "include_glossary": true }
  }')
echo "${SESSION}"
SESSION_ID=$(echo "${SESSION}" | python3 -c 'import json,sys; print(json.load(sys.stdin)["ai_session_id"])')
test -n "${SESSION_ID}" || { echo "session creation failed (model not pulled? see G-10 404 pitfall)"; exit 1; }

REPLY=$(curl -s -X POST "${EVG_URL}/v3/sessions/${SESSION_ID}/chat-messages" \
  "${AUTH[@]}" -F "user_message=How long do I have to return an item for a full refund?")
echo "${REPLY}" | python3 -c '
import json, sys
msg = json.load(sys.stdin)
text = " ".join(c["Text"] for c in msg["content"] if "Text" in c).lower()
assert "30 day" in text or "thirty day" in text, text
print("checkpoint 3 passed: grounded answer ->", text)
'
```

**You should see** — the session id, then something like
`checkpoint 3 passed: grounded answer -> full refunds are available within 30 days of purchase.`
**The first call can take a minute or more** while the backend loads the
model; the reply is not streamed here (see G-10 Step 3 for SSE streaming).

## Checkpoint 4 — Escalate to a human over Notifications

Now the customer says the quiet part out loud. The script's escalation rule
is a keyword match on the customer message — `escalate` or `manager` — and
the alert plane is a Notifications **topic**:

1. **Subscribe first** (public port `:40013`), writing the SSE stream to a
   file — this stands in for the support dashboard.
2. Publish a `support-escalations` event on the **loopback** port `:40014`
   (the publish route deliberately lives off the public router).
3. Assert the dashboard receives it.

```bash verify=auto
SSE_FILE=$(mktemp /tmp/t01-escalation.XXXXXX)
curl -s -N -X POST "${NOTIF_URL}/v3/notifications/subscribe/multi" \
  "${AUTH[@]}" -H "Content-Type: application/json" \
  -d "{\"tenant_id\": \"${TENANT_ID}\", \"subscriptions\": [{\"topic\": \"support-escalations\"}]}" \
  > "${SSE_FILE}" 2>&1 &
SSE_PID=$!
sleep 3
echo "dashboard subscribed (pid ${SSE_PID})"

QUESTION="I have asked twice already — escalate this to a manager."
case "$(echo "${QUESTION}" | tr '[:upper:]' '[:lower:]')" in
  *escalate*|*manager*) NEEDS_HUMAN=true ;;
  *) NEEDS_HUMAN=false ;;
esac
test "${NEEDS_HUMAN}" = "true" || { echo "escalation keyword not detected"; exit 1; }

PUB=$(curl -s -X POST "${NOTIF_PRIV_URL}/v3/notifications/publish" \
  "${AUTH[@]}" -H "Content-Type: application/json" \
  -d "{\"tenant_id\": \"${TENANT_ID}\", \"topic\": \"support-escalations\", \"sub_topic\": \"ticket-4711\", \"payload\": \"{\\\"question\\\": \\\"${QUESTION}\\\", \\\"priority\\\": \\\"high\\\"}\"}")
echo "${PUB}" | python3 -c 'import json,sys; n=json.load(sys.stdin)[0]; print("published:", n["topic"], n.get("sub_topic"))'

sleep 3
grep -q "support-escalations" "${SSE_FILE}" \
  && echo "checkpoint 4 passed: dashboard received the escalation" \
  || { echo "no escalation received"; kill "${SSE_PID}" 2>/dev/null; exit 1; }
kill "${SSE_PID}" 2>/dev/null
```

**You should see** — `dashboard subscribed`, `published: support-escalations
ticket-4711`, then `checkpoint 4 passed: dashboard received the escalation`.
The SSE event carries `topic`, `sub_topic`, and the JSON `payload` — the
dashboard would render it as a ticket card.

> [!NOTE]
> Publish lands on the **loopback** port (`:40014`) while dashboards
> subscribe on the **public** port (`:40013`) — both routers share one
> in-process broker, so a publish reaches every live subscription of the
> tenant. Keeping `publish` off the public router means untrusted callers
> can *receive* notifications but can never *inject* them.

## Checkpoint 5 — Failure injection: kill RAG, watch the assistant degrade, recover

A support assistant that silently hallucinates when its knowledge base is
down is worse than no assistant. Prove the failure is **visible**:

```bash verify=auto
pkill -x rag-service && echo "rag-service killed"

DOWN_CODE=$(curl -s -o /tmp/t01-down.txt -w "%{http_code}" -m 5 \
  -X POST "${RAG_URL}/v3/search" "${AUTH[@]}" \
  -H "Content-Type: application/json" \
  -d "{\"collection_id\": \"${COLLECTION}\", \"question\": \"refund window?\"}" || echo "000")
echo "search while RAG down: ${DOWN_CODE}"
test "${DOWN_CODE}" = "000" -o "${DOWN_CODE}" -ge 500 \
  && echo "failure is visible (no silent wrong answers)"

source /tmp/inaya_env.sh
export INAYA_LABS_ENVIRONMENT=local
nohup /Users/test/projects/Software/InayaLabs/target/debug/rag-service > /tmp/rag-service.log 2>&1 &
for i in $(seq 1 20); do
  CODE=$(curl -s -m 2 -o /dev/null -w "%{http_code}" "${RAG_URL}/v3/health" 2>/dev/null || echo 000)
  echo "rag health poll ${i}: ${CODE}"
  [ "${CODE}" = "200" ] && break
  sleep 2
done
test "${CODE}" = "200" || { echo "rag-service did not come back"; exit 1; }

UP_HIT=$(curl -sf -X POST "${RAG_URL}/v3/search" "${AUTH[@]}" \
  -H "Content-Type: application/json" \
  -d "{\"collection_id\": \"${COLLECTION}\", \"question\": \"How long do I have to return an item for a full refund?\"}")
echo "${UP_HIT}" | python3 -c '
import json, sys
d = json.load(sys.stdin)
assert d["chunks"], "no chunks after recovery"
print("checkpoint 5 passed: knowledge base recovered, corpus intact (restart does not wipe the store)")
'
```

**You should see** — `search while RAG down: 000` (connection refused),
`failure is visible`, health polls climbing to `200`, then
`checkpoint 5 passed: knowledge base recovered, corpus intact` — the FAQ
survives the restart because the RAG store is on disk, not in memory.

> [!TIP]
> In production you would not restart by hand: the platform's deployment
> layer re-schedules the replica, and your app retries with backoff (see
> [G-03 Errors, retries & idempotency](../guides/errors-retries-idempotency.html))
> — the sequence above is exactly what a retry loop experiences.

## Ship it

- **Production access:** swap the QA login for the real flows (G-01) —
  passkeys or federated sign-in — and point the base URLs at the deployed
  hosts (`*-sandbox.inayalabs.com` today; your own domain in production). The
  key pair becomes a real tenant credential issued by the tenant admin.
- **Model choice:** replace the pinned `Qwen3_VL_4B_Q4` with capability
  criteria (context length, parameter count — G-10's TIP) so the platform
  picks the best pulled model per request.
- **Escalation logic:** keyword matching is the tutorial's placeholder;
  Evergreen AI can classify intent (route the same chat through a second
  session with a classifier system prompt) before publishing.
- **Agentic operation:** every service in this tutorial is MCP-enabled —
  an agent can discover and drive the same flow (G-02); the platform's
  `llms.txt` indexes the contracts for agent discovery platforms.

## Cleanup

```bash
# end the chat session (G-10 Step 5), then remove the corpus:
curl -s -o /dev/null -w "%{http_code}\n" -X DELETE "${EVG_URL}/v3/sessions/${SESSION_ID}" "${AUTH[@]}"
curl -s -o /dev/null -w "%{http_code}\n" -X DELETE "${RAG_URL}/v3/documents/${DOC_ID}" "${AUTH[@]}"
```

**Expected** — both `204`s (see G-11 for the RAG delete semantics: `200`
for an unknown id, so treat this as best-effort cleanup, not a check).

## Next journeys

- **Build a social app backend** (T-02) — the Communication stack: users, profiles, feed, and DMs.
- **Choosing Evergreen vs Bella vs Chloe** (G-13) — swap the assistant's brain for a managed persona or a council.
- **Agentic inventory audit** (T-05) — drive this whole flow from an agent instead of a script.

### T-02 · Build a social app backend
URL: /docs/tutorials/social-app-backend.html

> **Scenario.** *Commons* is a community app for a neighborhood association.
> You will stand up its entire backend in one sitting: two residents sign in,
> set up profiles, publish to a shared feed, react and reply, and slide into
> DMs — across three CQRS service pairs (Users, Feed, Messaging) with their
> workflow services doing moderation and translation asynchronously.

This tutorial **composes** three guides instead of repeating them: [G-30 Managing users & profiles](../guides/managing-users-profiles.html) (identity), [G-40 Publishing and reading a feed](../guides/publishing-reading-feed.html), and [G-41 Direct messaging](../guides/direct-messaging-users.html). It adds what those guides can't show alone: **one coherent app** where the same identities flow through profiles → feed → DMs, and what happens to the app when an asynchronous pipeline dies mid-story.

**Time:** ~60 min · **Level:** intermediate

## Architecture

```text
                 ┌──────────── Auth :40004 ───────────┐
                 │  sign-in mints JWT + user record    │
                 └───────┬───────────────────┬────────┘
                         │                   │
        ┌────────────────▼───────┐   ┌───────▼────────────────┐
        │ Users Command :40171    │   │ Users Query :40170      │
        │  profile writes (PUT)   │   │  reads, search, batch   │
        └───────┬────────────────┘   └────────────────────────┘
                │ one identity, three surfaces
   ┌────────────▼─────────────┐   ┌──────────────────────────────┐
   │ Feed Command :40167       │   │ Messaging Command :40163      │
   │  post / reply / react     │   │  conversations / messages     │
   └───────┬───────────────────┘   └───────┬──────────────────────┘
           │ async (CQRS)                    │ async (CQRS)
   ┌───────▼───────────────────┐   ┌────────▼─────────────────────┐
   │ Feed Workflow :40169       │   │ Messaging Workflow :40165     │
   │  moderation → publish      │   │  moderation → publish         │
   └───────┬───────────────────┘   └────────┬─────────────────────┘
   ┌───────▼───────────────────┐   ┌────────▼─────────────────────┐
   │ Feed Query :40168          │   │ Messaging Query :40164        │
   │  timeline, search          │   │  threads, receipts            │
   └────────────────────────────┘   └──────────────────────────────┘
```

The shape to internalize: **writes are accepted immediately; reads become
true only after the workflow publishes.** Every "poll until visible" in this
tutorial is that contract in action.

## Prerequisites

- The local stack running: `auth-service` (`:40004`), the Users pair
  (`:40171`/`:40170`), the Feed trio (`:40167`/`:40168`/`:40169`), the
  Messaging trio (`:40163`/`:40164`/`:40165`), and **Postgres** (the CQRS
  services' store).
- The tenant key pair in `INAYA_API_KEY` / `INAYA_APP_KEY` and a tenant id in
  `TENANT_ID`. Any tenant id works on the local stack — the verification runs
  used `qa-tenant-t02`.
- Run all snippets **from the repository root** (`/Users/test/projects/Software/InayaLabs`).
- Polling loops use 5 s intervals; feed entries clear AI moderation at
  **~1–5 minutes each** on local hardware (messaging is much faster — the
  moderation pass is lighter), so the loops allow 3 minutes per feed
  checkpoint and 5 for the post-restart drain.

> [!PITFALL]
> Feed and Messaging content is encrypted **per tenant**: both the command
> services (on write) and the workflow services (on moderation) fetch the
> tenant's `feed_encryption_key_active` / `messaging_encryption_key_active`
> from Tenant Admin. A tenant that has never been registered there — or that
> has no active keys — makes **every feed/messaging call fail with a generic
> 500** (`An internal server error occurred`). If you create a fresh tenant
> for this tutorial, provision it first:
>
> ```bash
> # on the tenant-admin public router (:40173), with the tenant key pair
> # headers, a JWT, and two hex-encoded AES-256 keys (64 hex chars each):
> curl -X POST "http://127.0.0.1:40173/v3/tenants" \
>   -H "x-api-key: ${INAYA_API_KEY}" -H "x-client-app-key: ${INAYA_APP_KEY}" \
>   -H "Authorization: Bearer ${ADMIN_JWT}" -H "Content-Type: application/json" \
>   -d "{\"id\": \"${TENANT_ID}\", \"environment_name\": \"local\"}"
>
> curl -X PATCH "http://127.0.0.1:40173/v3/tenant-platform-options/tenant/${TENANT_ID}" \
>   -H "x-api-key: ${INAYA_API_KEY}" -H "x-client-app-key: ${INAYA_APP_KEY}" \
>   -H "Authorization: Bearer ${ADMIN_JWT}" -H "Content-Type: application/json" \
>   -d "{\"feed_encryption_key_active\": \"<64-hex-chars>\", \
>      \"messaging_encryption_key_active\": \"<64-hex-chars>\"}"
> ```
>
> (Generate keys with `openssl rand -hex 32`.) Users/Auth do **not** need
> this — only content-encrypting services do — which is why a half-provisioned
> tenant fails in confusing order: sign-in works, posting 500s.

## Companion code

```text
InayaLabs-website/docs-src/tutorials/_code/social-app-backend/
├── social_app.sh   ← the assembled driver (all checkpoints)
└── README.md       ← how to run it standalone
```

Download links: [social_app.sh](_code/social-app-backend/social_app.sh) · [README.md](_code/social-app-backend/README.md)

## Checkpoint 1 — Two residents sign in

Every persona needs a JWT; the login response also carries the user's
platform id (`user.id`) — you will need it to address DMs later:

```bash verify=auto
KEYS=(-H "x-api-key: ${INAYA_API_KEY}" -H "x-client-app-key: ${INAYA_APP_KEY}")
sign_in() {  # username -> "JWT USERID"
  curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" "${KEYS[@]}" \
    -H "Content-Type: application/json" \
    -d "{\"device_id\": \"t02-device-01\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"$1\"}" \
    | python3 -c 'import json,sys; d=json.load(sys.stdin); print(d["token"], d["user"]["id"])'
}
read PRIYA_T PRIYA_U <<< "$(sign_in t02-priya)"
read OMAR_T  OMAR_U  <<< "$(sign_in t02-omar)"
echo "priya: ${#PRIYA_T} chars, id ${PRIYA_U:0:12}…"
echo "omar:  ${#OMAR_T} chars, id ${OMAR_U:0:12}…"
test -n "${PRIYA_T}" && test -n "${OMAR_T}" && test -n "${OMAR_U}" \
  && echo "checkpoint 1 passed: two identities, two JWTs, omar's id captured"
```

**You should see** — `checkpoint 1 passed: two identities, two JWTs, omar's
id captured`. (G-01 covers production sign-in; the QA shortcut stands in here.)

## Checkpoint 2 — Profiles: born empty, fleshed out by their owner

Omar's user record was created automatically at first sign-in — read it,
modify it locally, and PUT the whole DTO back (the get-modify-put contract,
G-30). Then Priya discovers him via user search:

```bash verify=auto
USQ_AUTH=(-H "Authorization: Bearer ${OMAR_T}" "${KEYS[@]}")
curl -sf "${USQ_URL}/v3/users/${OMAR_U}" "${USQ_AUTH[@]}" > /tmp/t02-omar.json

python3 - <<'EOF'
import json
me = json.load(open("/tmp/t02-omar.json"))
me["first_name"] = "Omar"
me["last_name"] = "Haddad"
me["display_name"] = "Omar H."
me["bio"] = "Plot 14 — tomatoes and one very opinionated rooster."
me["location_display_name"] = "North Garden"
me["profile_json"] = {"plot": 14, "skills": ["irrigation", "compost"]}
json.dump(me, open("/tmp/t02-omar-updated.json", "w"))
EOF

curl -sf -o /dev/null -X PUT "${USC_URL}/v3/users/${OMAR_U}" \
  -H "Authorization: Bearer ${OMAR_T}" "${KEYS[@]}" \
  -H "Content-Type: application/json" \
  --data-binary @/tmp/t02-omar-updated.json

curl -sf "${USQ_URL}/v3/users/${OMAR_U}" "${USQ_AUTH[@]}" \
  | python3 -c 'import json,sys; u=json.load(sys.stdin); print("omar is now:", u["display_name"], "|", u["bio"][:40])'

curl -sf -X POST "${USQ_URL}/v3/search" \
  -H "Authorization: Bearer ${PRIYA_T}" "${KEYS[@]}" \
  -H "Content-Type: application/json" \
  -d '{"search_criteria":[{"name":"Omar"}]}' \
  | python3 -c 'import json,sys; r=json.load(sys.stdin); us = r["users"] if isinstance(r, dict) else r; assert any("Omar" in json.dumps(u) for u in us), r; print("checkpoint 2 passed: priya found omar via user search")'
```

**You should see** — `omar is now: Omar H. | Plot 14 — tomatoes and one very
opinionated ro…`, then `checkpoint 2 passed: priya found omar via user
search`. Note the search response shape is version-dependent — the snippet
accepts either a bare list or a `{"users": [...]}` envelope; assert on
*content*, not container.

> [!PITFALL]
> Profile writes are **self-only**: `PUT /v3/users/{id}` with another user's
> JWT returns 401, and the DTO round-trips whole — partial patches are not
> supported (G-30).

## Checkpoint 3 — The feed: post, reply, poll, search

Priya posts to the neighborhood feed; Omar replies. Both writes return ids
immediately, but **the entries only become readable after the Feed Workflow
publishes them** — that is the CQRS contract you poll on. Budget real time
for the poll: each entry clears an AI moderation pass inside the workflow,
which on local hardware takes **~1–5 minutes per entry** (small local
models, cold caches, per-entry Evergreen sessions). Two entries can
therefore take several minutes to surface, and the loops allow 3:

```bash verify=auto
POST_ID=$(curl -s -X POST "${FDC_URL}/v3/feed/post" \
  -H "Authorization: Bearer ${PRIYA_T}" "${KEYS[@]}" \
  -H "Content-Type: application/json" \
  -d '{"content": "Seedling swap this Saturday at the tool shed — bring extras.", "languageCode": "en-US", "userHashtags": ["seedling-swap"], "mediaFiles": []}' \
  | python3 -c 'import json,sys; print(json.load(sys.stdin))')
echo "post id: ${POST_ID}"

REPLY_ID=$(curl -s -X POST "${FDC_URL}/v3/feed/reply" \
  -H "Authorization: Bearer ${OMAR_T}" "${KEYS[@]}" \
  -H "Content-Type: application/json" \
  -d "{\"parentId\": \"${POST_ID}\", \"content\": \"Saving two trays of Roma tomatoes for you.\", \"languageCode\": \"en-US\", \"userHashtags\": [], \"mediaFiles\": []}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin))')
echo "reply id: ${REPLY_ID}"

SEEN=""
for i in $(seq 1 36); do
  SEEN=$(curl -s "${FDQ_URL}/v3/feed?page=1&page_size=10" \
    -H "Authorization: Bearer ${OMAR_T}" "${KEYS[@]}" \
    | python3 -c 'import json,sys
es = json.load(sys.stdin)["entries"]
print("yes" if any(e["id"] == "'"${POST_ID}"'" for e in es) else "")')
  [ "${SEEN}" = "yes" ] && { echo "published after ${i} poll(s)"; break; }
  sleep 5
done
test "${SEEN}" = "yes" || { echo "post never published"; exit 1; }

curl -s -X POST "${FDQ_URL}/v3/feed/search?page=1&page_size=10" \
  -H "Authorization: Bearer ${OMAR_T}" "${KEYS[@]}" \
  -H "Content-Type: application/json" \
  -d '{"hashtag": "seedling-swap"}' \
  | python3 -c 'import json,sys; es=json.load(sys.stdin)["entries"]; assert any("swap" in e["content"] for e in es), es; print("checkpoint 3 passed: feed round trip (post + reply visible, hashtag search hit)")'
```

**You should see** — post and reply ids, `published after N poll(s)`, then
`checkpoint 3 passed: feed round trip (post + reply visible, hashtag search
hit)`.

## Checkpoint 4 — DMs: conversation, message, receipts

Omar slides into Priya's DMs about the tomato trays. Conversation creation
is **idempotent** (same participants + title returns the same conversation),
the message lands *pending* (`content: null`), and the workflow publishes it
seconds later. Receipts close the loop:

```bash verify=auto
CONV_BODY=$(printf '{"otherParticipants": ["%s"], "title": "Seedling swap"}' "${PRIYA_U}")
CID=$(curl -sf -X POST "${MSC_URL}/v3/messaging/conversations" \
  -H "Authorization: Bearer ${OMAR_T}" "${KEYS[@]}" \
  -H "Content-Type: application/json" -d "${CONV_BODY}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["id"])')
CID2=$(curl -sf -X POST "${MSC_URL}/v3/messaging/conversations" \
  -H "Authorization: Bearer ${OMAR_T}" "${KEYS[@]}" \
  -H "Content-Type: application/json" -d "${CONV_BODY}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["id"])')
test "${CID}" = "${CID2}" && echo "conversation ${CID:0:12}… (idempotent re-create)"

MSG_BODY=$(printf '{"conversationId": "%s", "content": "Could I get three of the Roma trays?", "mediaUrls": []}' "${CID}")
MID=$(curl -sf -X POST "${MSC_URL}/v3/messaging/messages" \
  -H "Authorization: Bearer ${OMAR_T}" "${KEYS[@]}" \
  -H "Content-Type: application/json" -d "${MSG_BODY}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["id"])')
echo "message ${MID:0:12}… (pending until the workflow publishes)"

VISIBLE=""
for i in $(seq 1 36); do
  VISIBLE=$(curl -s "${MSQ_URL}/v3/messaging/conversations/${CID}/messages?page=1&page_size=10" \
    -H "Authorization: Bearer ${PRIYA_T}" "${KEYS[@]}" \
    | python3 -c 'import json,sys
ms = json.load(sys.stdin)
m = next((m for m in ms if m["id"] == "'"${MID}"'"), None)
print((m["content"] or "") if m else "")')
  grep -q "Roma trays" <<< "${VISIBLE}" && { echo "published after ${i} poll(s)"; break; }
  sleep 5
done
grep -q "Roma trays" <<< "${VISIBLE}" || { echo "message never published"; exit 1; }

curl -sf -o /dev/null -X POST "${MSC_URL}/v3/messaging/messages/${MID}/delivery-receipts" \
  -H "Authorization: Bearer ${PRIYA_T}" "${KEYS[@]}"
curl -sf -o /dev/null -X POST "${MSC_URL}/v3/messaging/messages/${MID}/read-receipts" \
  -H "Authorization: Bearer ${PRIYA_T}" "${KEYS[@]}"
curl -s "${MSQ_URL}/v3/messaging/messages/${MID}/receipts?page=1&page_size=10" \
  -H "Authorization: Bearer ${OMAR_T}" "${KEYS[@]}" \
  | python3 -c 'import json,sys
rs = json.load(sys.stdin)
assert any(r["status"] == "DeliveredAndRead" for r in rs), rs
print("checkpoint 4 passed: priya read the DM (DeliveredAndRead visible to omar)")'
```

**You should see** — the idempotent conversation line, `published after N
poll(s)`, then `checkpoint 4 passed: priya read the DM (DeliveredAndRead
visible to omar)`.

## Checkpoint 5 — Failure injection: kill the feed pipeline mid-story

Now the lesson only a composed tutorial can teach. Kill `feed-workflow-service`
— the moderation/publish pipeline — and watch the **write side keep
accepting work while the read side goes quiet**. Nothing is lost; it is
queued behind the dead pipeline. After the restart, give the drain real
time: the workflow re-resolves tenant keys and re-runs moderation per
queued entry, so on local hardware the stalled post can surface **anywhere
from ~1 to ~7 minutes** after the restart — the poll allows 5 minutes and
the final assertion (not the clock) is what guarantees correctness:

```bash verify=auto
pkill -x feed-workflow-service && echo "feed-workflow-service killed"

STALL_ID=$(curl -s -X POST "${FDC_URL}/v3/feed/post" \
  -H "Authorization: Bearer ${PRIYA_T}" "${KEYS[@]}" \
  -H "Content-Type: application/json" \
  -d '{"content": "STALL TEST: rain moved the swap to Sunday.", "languageCode": "en-US", "userHashtags": ["seedling-swap"], "mediaFiles": []}' \
  | python3 -c 'import json,sys; print(json.load(sys.stdin))')
echo "write accepted while pipeline is down: ${STALL_ID}"

STALLED="yes"
for i in $(seq 1 6); do
  FOUND=$(curl -s "${FDQ_URL}/v3/feed?page=1&page_size=20" \
    -H "Authorization: Bearer ${OMAR_T}" "${KEYS[@]}" \
    | python3 -c 'import json,sys
es = json.load(sys.stdin)["entries"]
print("yes" if any(e["id"] == "'"${STALL_ID}"'" for e in es) else "")')
  [ "${FOUND}" = "yes" ] && { STALLED="no"; break; }
  sleep 5
done
test "${STALLED}" = "yes" && echo "checkpoint 5a passed: read side is quiet while the pipeline is down (no error, no data — just lag)"

source /tmp/inaya_env.sh
export INAYA_LABS_ENVIRONMENT=local
nohup /Users/test/projects/Software/InayaLabs/target/debug/feed-workflow-service > /tmp/feed-workflow-service.log 2>&1 &
RECOVERED=""
for i in $(seq 1 60); do
  FOUND=$(curl -s "${FDQ_URL}/v3/feed?page=1&page_size=20" \
    -H "Authorization: Bearer ${OMAR_T}" "${KEYS[@]}" \
    | python3 -c 'import json,sys
es = json.load(sys.stdin)["entries"]
print("yes" if any(e["id"] == "'"${STALL_ID}"'" for e in es) else "")')
  [ "${FOUND}" = "yes" ] && { RECOVERED="yes"; echo "published after restart, poll ${i}"; break; }
  sleep 5
done
test "${RECOVERED}" = "yes" || { echo "stalled post never recovered"; exit 1; }

curl -s "${FDQ_URL}/v3/feed?page=1&page_size=20" \
  -H "Authorization: Bearer ${OMAR_T}" "${KEYS[@]}" \
  | python3 -c 'import json,sys
es = json.load(sys.stdin)["entries"]
assert any("swap to Sunday" in e["content"] for e in es), [e["content"] for e in es]
print("checkpoint 5b passed: queued post drained intact — CQRS lag, not data loss")'
```

**You should see** — `write accepted while pipeline is down`, then
`checkpoint 5a passed: read side is quiet…`, then `published after restart,
poll N`, and `checkpoint 5b passed: queued post drained intact — CQRS lag,
not data loss`.

> [!NOTE]
> This is the exact failure shape a monitoring alert should key on: **write
> successes without corresponding read-side arrivals**. Alert on pipeline
> lag (oldest unpublished entry age), not on command-service health — the
> command side stayed healthy through the whole outage.

## Ship it

- **Production access:** swap the QA login for the real flows (G-01); point
  base URLs at the deployed hosts. The three CQRS pairs scale independently
  — reads (Query services) can be replicated far beyond writes (Command).
- **Realtime instead of polling:** the workflow services also emit
  Notifications events on publication — swap the poll loops for SSE
  subscriptions (T-01 Checkpoint 4 shows the pattern).
- **Media:** feed posts and messages both accept media references — stage
  files through the CDN (G-04/G-71) and pass the resulting paths in
  `mediaFiles`/`mediaUrls`.
- **Moderation posture:** the workflow pipeline is where content policy
  lives. A rejected post never reaches the Query side; surface rejection to
  authors via the Notifications topic pattern from T-01.

## Cleanup

```bash
# best-effort; these CQRS delete semantics are covered in G-40/G-41
curl -s -o /dev/null -w "post delete: %{http_code}\n" -X DELETE \
  "${FDC_URL}/v3/feed/post/${POST_ID}" -H "Authorization: Bearer ${PRIYA_T}" "${KEYS[@]}"
curl -s -o /dev/null -w "message delete: %{http_code}\n" -X DELETE \
  "${MSC_URL}/v3/messaging/messages/${MID}" -H "Authorization: Bearer ${OMAR_T}" "${KEYS[@]}"
```

**Expected** — both `204`s (feed delete is idempotent; message repeat-delete
returns 4xx — see G-40/G-41 for the exact contracts).

## Next journeys

- **Build a support assistant** (T-01) — the other multi-service composition: Auth → RAG → Evergreen → Notifications.
- **Finding matches between profiles** (G-31) — turn user profiles into recommendations.
- **Uploading files the platform way** (G-04) — media for feed posts and DMs.

### T-03 · Build a media pipeline
URL: /docs/tutorials/media-pipeline.html

> **Scenario.** *Fieldnote* is a capture-and-publish app: users upload photos
> and short clips from the field, the platform derives web-friendly
> derivatives (thumbnails, trimmed clips), and a manifest of each asset's
> derivatives is cached for fast reads. You will build that pipeline across
> four services: **CDN** (durable object storage), **Image Transformer** and
> **Video Transformer** (derivative workers), and **Memcache** (the
> derivative manifest).

The shape to internalize: **the CDN is the only source of truth; everything
downstream is re-derivable.** Transformers read from shared storage and
return fresh bytes; memcache holds nothing that cannot be rebuilt.

**Time:** ~30 min · **Level:** beginner

## Architecture

```text
                 ┌──────────── Auth :40004 ─────────────┐
                 │  one JWT, reused by every service     │
                 └───────┬───────────────┬───────────────┘
                         │               │
        ┌────────────────▼───┐      ┌────▼───────────────────┐
        │ CDN :40040          │      │ Memcache :40011         │
        │ durable objects     │      │ derivative manifest     │
        │ (shared storage)    │      │ (in-memory, cache only) │
        └───────┬─────────────┘      └────────▲───────────────┘
                │ source_path                │ add / get / delete
   ┌────────────▼──────────────┐             │
   │ Image Transformer :40030   │  derivatives│
   │  resize · transcode        ├─────────────┘
   ├────────────────────────────┤
   │ Video Transformer :40160   │
   │  trim · transcode · mux    │
   └────────────────────────────┘
```

Two placement rules make this work on one host:

- The CDN and both transformers **share a filesystem**: the CDN's
  `INAYA_LABS_CDN_SHARED_STORAGE_PATH` is visible to the transformers, so a
  transformer can open an uploaded file by absolute path.
- Video job outputs land in `INAYA_LABS_VIDEO_OUTPUT_DIR` (default: the
  system temp dir). Set it to a stable path so the tutorial can find what
  the job produced.

## Prerequisites

- The local stack running: `auth-service` (`:40004`), `cdn` (`:40040`),
  `image-transformer-service` (`:40030`), `video-transformer-service`
  (`:40160`), `memcache-service` (public `:40011`), plus `ffmpeg` and
  `python3` with Pillow (both used to *make test assets* — the platform
  calls themselves are plain HTTP).
- Boot the media services with the shared storage root pinned:

  ```bash
  source /tmp/inaya_env.sh
  export INAYA_LABS_ENVIRONMENT=local
  export INAYA_LABS_CDN_SHARED_STORAGE_PATH=/tmp/inaya-cdn-storage
  export INAYA_LABS_VIDEO_OUTPUT_DIR=/tmp/t03-video-output
  nohup target/debug/cdn                        > /tmp/cdn-service.log 2>&1 &
  nohup target/debug/image-transformer-service  > /tmp/image-transformer-service.log 2>&1 &
  nohup target/debug/video-transformer-service  > /tmp/video-transformer-service.log 2>&1 &
  ```

- The tenant key pair in `INAYA_API_KEY` / `INAYA_APP_KEY`, a tenant id in
  `TENANT_ID` (any works locally; verification used `qa-tenant-t02`), and a
  JWT (G-01 / the QA sign-in shortcut — every checkpoint below refreshes
  one).
- Run all snippets **from the repository root** (`/Users/test/projects/Software/InayaLabs`).

> [!PITFALL]
> Every service here — CDN *and* both transformers *and* memcache — requires
> the tenant key-pair headers (`x-api-key` / `x-client-app-key`), and every
> endpoint except transformer health checks also requires a **valid JWT**,
> including `GET /v3/cdn/files/download?token=…`. The download token
> authorizes the *path*; the JWT authorizes the *caller*. A token-only
> download attempt 401s — that is the contract, not a bug.

## Companion code

```text
InayaLabs-website/docs-src/tutorials/_code/media-pipeline/
├── media_pipeline.sh   ← the assembled driver (all checkpoints)
└── README.md           ← how to run it standalone
```

Download links: [media_pipeline.sh](_code/media-pipeline/media_pipeline.sh) · [README.md](_code/media-pipeline/README.md)

## Checkpoint 1 — Stage an asset in the CDN and verify it round-trips

First make a test photo (any 1200×800 PNG — this stands in for a user
upload), then stage it in the CDN. Upload is multipart: a `path` text field
(namespaced per tenant) and a `file` bytes field:

```bash verify=auto
KEYS=(-H "x-api-key: ${INAYA_API_KEY}" -H "x-client-app-key: ${INAYA_APP_KEY}")
sign_in() {
  curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" "${KEYS[@]}" \
    -H "Content-Type: application/json" \
    -d "{\"device_id\": \"t03-device-01\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"$1\"}" \
    | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])'
}
TOKEN=$(sign_in t03-priya)

python3 - <<'EOF'
from PIL import Image
img = Image.new("RGB", (1200, 800), (30, 90, 160))
for x in range(0, 1200, 100):
    for y in range(0, 800, 100):
        img.putpixel((x, y), (255, 200, 40))
img.save("/tmp/t03-hero.png")
print("test asset: /tmp/t03-hero.png")
EOF

curl -sf -o /dev/null -w "upload: HTTP %{http_code}\n" \
  -X POST "${CDN_URL}/v3/cdn/files/upload?overwrite=true" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}" \
  -F "path=t03/hero.png" -F "file=@/tmp/t03-hero.png"
```

**You should see** — `upload: HTTP 201`. The `overwrite=true` query param
matters on re-runs: without it, uploading to an existing path 409s.

Now mint a download token and fetch the bytes back. Note the JWT rides
alongside the token:

```bash verify=auto
TOK=$(curl -sf -X POST "${CDN_URL}/v3/cdn/files/token" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}" \
  -H "Content-Type: application/json" \
  -d '{"path": "t03/hero.png"}')
echo "token: ${TOK:0:60}…"
DL_TOKEN=$(python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])' <<< "${TOK}")

curl -sf "http://127.0.0.1:40040/v3/cdn/files/download?token=${DL_TOKEN}" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}" \
  -o /tmp/t03-download.png
cmp /tmp/t03-hero.png /tmp/t03-download.png \
  && echo "checkpoint 1 passed: staged asset downloads byte-identical via token"
```

**You should see** — a token payload, then
`checkpoint 1 passed: staged asset downloads byte-identical via token`.

> [!PITFALL]
> Calling `download?token=…` **without** the JWT header returns 401 —
> `{"error":"Missing Authorization token"}`. Tokens scope *what* may be
> downloaded; the JWT says *who* is downloading.

## Checkpoint 2 — Derive an image derivative

The Image Transformer takes a **local absolute `source_path`** — not a CDN
URL. The uploaded file already lives on the shared filesystem under the
CDN's storage root; locate it and resize:

```bash verify=auto
SRC=$(find "${CDN_ROOT}/files" -name "hero.png" | head -1)
test -n "${SRC}" && echo "source: ${SRC}"

curl -sf -o /tmp/t03-thumb.png \
  -X POST "${IMG_URL}/v3/images/resize" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}" \
  -H "Content-Type: application/json" \
  -d "{\"source_path\": \"${SRC}\", \"tenant_id\": \"${TENANT_ID}\", \"mode\": \"Single\", \"width\": 300, \"height\": 200}"

python3 - <<'EOF'
from PIL import Image
size = Image.open("/tmp/t03-thumb.png").size
assert size == (300, 200), size
print("checkpoint 2 passed: thumbnail derived at 300x200 (binary PNG response)")
EOF
```

**You should see** — the absolute path under the storage root, then
`checkpoint 2 passed: thumbnail derived at 300x200 (binary PNG response)`.

> [!NOTE]
> The transformer response **is the file** (content-typed binary), not a
> job id — image work is synchronous and fast. Video work is not (see
> Checkpoint 3). Also: re-runs of this tutorial reuse the same
> `find` — the storage layout hashes tenant and namespace into directory
> names, so never construct these paths by hand.

Finish the loop: stage the derivative back in the CDN, next to the original:

```bash verify=auto
curl -sf -o /dev/null -w "derivative staged: HTTP %{http_code}\n" \
  -X POST "${CDN_URL}/v3/cdn/files/upload?overwrite=true" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}" \
  -F "path=t03/hero-thumb.png" -F "file=@/tmp/t03-thumb.png"
```

**You should see** — `derivative staged: HTTP 201`.

## Checkpoint 3 — Derive a video clip (async job)

Video work is a **202 + job resource** pattern: the service accepts the job,
processes it on a worker task, and completion is announced on a
notifications topic. Make a 6-second stand-in clip, stage it, and trim
seconds 1–4 off it:

```bash verify=auto
ffmpeg -y -loglevel error \
  -f lavfi -i testsrc=duration=6:size=320x240:rate=10 -pix_fmt yuv420p \
  /tmp/t03-clip.mp4

curl -sf -o /dev/null -w "clip staged: HTTP %{http_code}\n" \
  -X POST "${CDN_URL}/v3/cdn/files/upload?overwrite=true" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}" \
  -F "path=t03/clip.mp4" -F "file=@/tmp/t03-clip.mp4"

VID_SRC=$(find "${CDN_ROOT}/files" -name "clip.mp4" | head -1)
JOB=$(curl -sf -X POST "${VID_URL}/v3/video/trim" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}" \
  -H "Content-Type: application/json" \
  -d "{\"source_path\": \"${VID_SRC}\", \"tenant_id\": \"${TENANT_ID}\", \"mode\": \"segment\", \"in_point\": \"00:00:01\", \"out_point\": \"00:00:04\"}")
JOB_ID=$(python3 -c 'import json,sys; d=json.load(sys.stdin); print(d["id"]); assert d["status"] in ("pending","running"), d' <<< "${JOB}")
echo "job ${JOB_ID:0:16}… accepted (202, status pending)"

OUT_FILE="${VID_OUT}/${TENANT_ID}/${JOB_ID}.mp4"
for i in $(seq 1 24); do
  [ -f "${OUT_FILE}" ] && { echo "job finished after ${i} poll(s)"; break; }
  sleep 5
done
test -f "${OUT_FILE}" || { echo "job never produced output"; exit 1; }

DUR=$(ffprobe -v error -show_entries format=duration -of csv=p=0 "${OUT_FILE}")
python3 -c "d=float('${DUR}'); assert 0 < d < 6, d; print(f'checkpoint 3 passed: trimmed clip is {d:.1f}s (source was 6.0s)')"
```

**You should see** — `clip staged: HTTP 201`, the job accepted line, `job
finished after N poll(s)`, then `checkpoint 3 passed: trimmed clip is X.Xs
(source was 6.0s)`.

> [!PITFALL]
> Two video-contract sharp edges: (1) enum values deserialize **snake_case**
> — `"segment"`, `"start"`, `"end"` — and `"Segment"` is rejected with a
> 422. (2) There is no REST endpoint to poll the job: the 202 response is
> the last REST word, and completion arrives as a **Notifications event**
> (`VideoJobPending` topic, job id as sub-topic — T-01 Checkpoint 4 shows
> the SSE subscribe pattern). Polling the output file works everywhere,
> which is what the snippet does; a keyframe-aligned trim can also run
> slightly long, so assert a *range*, never an exact duration.

## Checkpoint 4 — Cache the derivative manifest

Now the read-side fast path. The manifest — which derivatives exist for an
asset — goes into memcache as one JSON value per asset:

```bash verify=auto
MANIFEST='[{"kind":"thumb","path":"t03/hero-thumb.png","width":300,"height":200},{"kind":"clip-trim","path":"video-output/'"${JOB_ID}"'.mp4","in":"00:00:01","out":"00:00:04"}]'
python3 - "$TENANT_ID" "$MANIFEST" <<'EOF' > /tmp/t03-manifest-body.json
import json, sys
print(json.dumps({"key": "t03:manifest:hero", "value": sys.argv[2], "ttl_in_seconds": 3600}))
EOF

curl -sf -o /dev/null -w "manifest cached: HTTP %{http_code}\n" \
  -X POST "${MEM_URL}/v3/mem-cache/add/${TENANT_ID}" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}" \
  -H "Content-Type: application/json" \
  --data-binary @/tmp/t03-manifest-body.json

GOT=$(curl -sf "${MEM_URL}/v3/mem-cache/get/${TENANT_ID}/t03:manifest:hero" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}")
python3 - "$GOT" <<'EOF'
import json, sys
entries = json.loads(sys.argv[1])
assert any(e["kind"] == "thumb" and e["width"] == 300 for e in entries), entries
assert any(e["kind"] == "clip-trim" for e in entries), entries
print("checkpoint 4 passed: manifest round-trips with both derivatives listed")
EOF

curl -s -o /dev/null -w "unknown key: HTTP %{http_code}\n" \
  "${MEM_URL}/v3/mem-cache/get/${TENANT_ID}/t03:manifest:nope" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}"
```

**You should see** — `manifest cached: HTTP 201`, the checkpoint line, and
`unknown key: HTTP 404` (a miss is a clean 404, not an error).

> [!NOTE]
> `get` returns the **raw stored value** as the response body — not a JSON
> envelope — because memcache is payload-agnostic (values are AES-256
> encrypted at rest and decrypted on read; `CACHE_ENCRYPTION_KEY` from the
> environment). Store strings you can parse, and parse what you get back.

## Checkpoint 5 — Failure injection: the cache dies, the pipeline survives

Kill memcache and watch the failure shape: the **reads fail loudly**, while
the source of truth (the CDN) is untouched. Then restart and observe the
cache's real contract — it comes back **empty**:

```bash verify=auto
pkill -x memcache-service && echo "memcache-service killed"

curl -s -o /dev/null \
  --max-time 5 \
  "${MEM_URL}/v3/mem-cache/get/${TENANT_ID}/t03:manifest:hero" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}" \
  || echo "get during outage: connection refused (cache consumer must degrade)"

curl -sf -o /dev/null -w "cdn still serves: HTTP %{http_code}\n" \
  "http://127.0.0.1:40040/v3/cdn/files/download?token=${DL_TOKEN}" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}"
echo "checkpoint 5a passed: cache outage fails loudly; source of truth unaffected"

source /tmp/inaya_env.sh
export INAYA_LABS_ENVIRONMENT=local
nohup /Users/test/projects/Software/InayaLabs/target/debug/memcache-service > /tmp/memcache-service.log 2>&1 &
sleep 2

curl -s -o /dev/null -w "get after restart: HTTP %{http_code}\n" \
  "${MEM_URL}/v3/mem-cache/get/${TENANT_ID}/t03:manifest:hero" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}"

curl -sf -o /dev/null -w "manifest rebuilt: HTTP %{http_code}\n" \
  -X POST "${MEM_URL}/v3/mem-cache/add/${TENANT_ID}" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}" \
  -H "Content-Type: application/json" \
  --data-binary @/tmp/t03-manifest-body.json
curl -sf "${MEM_URL}/v3/mem-cache/get/${TENANT_ID}/t03:manifest:hero" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}" \
  | grep -q "hero-thumb.png" \
  && echo "checkpoint 5b passed: cache restarted empty — manifest re-derivable from CDN state, not lost"
```

**You should see** — `memcache-service killed`, either an HTTP error or the
`connection refused` line, `cdn still serves: HTTP 200`, then
`get after restart: HTTP 404` (the restart wiped the in-memory cache),
`manifest rebuilt: HTTP 201`, and the final checkpoint line.

> [!NOTE]
> This is the difference between the CQRS tutorials (T-02) and this one:
> there, the async pipeline held the only copy of *published* state, so a
> dead pipeline meant lag. Here the cache holds **derived** state — the
> worst a restart does is force re-derivation. Design caches to be
> rebuildable from the source of truth, and they can die whenever they
> like.

## Ship it

- **Production access:** point the CDN base URL at the deployed host and
  swap the local filesystem assumptions — in deployed environments the
  shared storage root is a mounted bucket, and transformers reach it the
  same way (absolute `source_path`).
- **Notifications instead of file-polling:** subscribe to the
  `VideoJobPending` topic (T-01 Checkpoint 4's SSE pattern) and let the
  workflow push completions instead of polling the output dir.
- **More derivatives:** the same loop covers `transcode` (formats), `mux`
  (combine clips), and `upscale` — and the Image Transformer's `Multi`
  resize mode derives a whole responsive set in one call.
- **Cleanup:** `DELETE /v3/cdn/files/{path}` removes staged assets when
  you're done.

## Copy agent prompt

> Follow tutorial T-03 "Build a media pipeline" at
> `/Users/test/projects/Software/InayaLabs/InayaLabs-website/docs/tutorials/media-pipeline.html`
> end to end. Stage both test assets, derive the image thumbnail and the
> trimmed clip, cache the manifest, then kill and restart memcache. Stop if
> any checkpoint assertion fails and report which one.

### T-04 · Build a multilingual voice assistant
URL: /docs/tutorials/multilingual-voice-assistant.html

> **Scenario.** *Amara* runs a community garden whose volunteers speak
> English, Spanish, and Arabic. She wants to ask a question out loud (well,
> typed — this tutorial keeps it scriptable), get an answer from the AI
> assistant, translate it for a volunteer, and **have it spoken back in her
> own voice** — so a shared device at the garden gate sounds like her in
> every language. You will compose three services: **Evergreen AI** (the
> brain), **Linga** (the translator), and **Speech** (voice print + TTS).

The shape to internalize: **the same text pipeline serves every language,
and the voice is data, not configuration.** A voice print registered once
rides along as a CDN path on every synthesis call.

**Time:** ~60 min · **Level:** advanced (long-running AI calls, model
warm-up, multipart audio)

## Architecture

```text
  you ──► ┌──────────────── Evergreen AI :40071 ────────────────┐
          │  session (criteria-picked model) + chat message      │
          └───────┬─────────────────────────────────────────────┘
                  │ answer text (JSON, ~20–60 s on local models)
                  ▼
          ┌──────────────── Linga :40065 ───────────────────────┐
          │  POST /v3/translation/translate                     │
          │  en-US ──► es-ES (via Evergreen, single-turn)       │
          └───────┬─────────────────────────────────────────────┘
                  │ translated text
                  ▼
  your voice ──► ┌────────────── Speech :40161 ──────────────────┐
  (WAV upload,   │  voice print  ──►  CDN path (registered once) │
   voice print)  │  text-to-speech {input, reference_voiceprint} │
                 └───────┬────────────────────────────────────────┘
                         │ audio/mpeg (your voice, Spanish words)
                         ▼
                       speaker
```

Hidden edge: **Speech's TTS also calls Linga internally** (language
detection on the input text). One service, two consumers — Checkpoint 5
makes that visible.

## Prerequisites

- The local stack running: `auth-service` (`:40004`),
  `evergreen-ai-service` (`:40071`), `linga-service` (`:40065`),
  `speech-service` (`:40161`), **Ollama** (Evergreen's inference backend —
  it must be up *before* evergreen boots, or evergreen's model inventory
  comes up empty), and the **CDN** (`:40040`) — Speech persists voice
  prints there.
- Boot speech with the smaller TTS model and a generous synthesis deadline
  (the 1.7B default exceeds the 120 s synthesis deadline on a loaded dev
  box; both knobs are env vars):

  ```bash
  source /tmp/inaya_env.sh
  export INAYA_LABS_ENVIRONMENT=local
  export INAYA_LABS_CDN_SHARED_STORAGE_PATH=/tmp/inaya-cdn-storage
  export QWEN_TTS_MODEL_DIR="$HOME/.InayaLabs/models/hf/Qwen3-TTS-12Hz-0.6B-Base"
  export TTS_SYNTH_DEADLINE_SECS=300
  nohup target/debug/speech-service > /tmp/speech-service.log 2>&1 &
  ```

- The tenant key pair in `INAYA_API_KEY` / `INAYA_APP_KEY`, a tenant id in
  `TENANT_ID` (verification used `qa-tenant-t02`).
- macOS `say` + `afconvert` (to make a test voice sample) and `ffprobe`
  (to validate audio). Any WAV with intelligible speech works for the
  voice print — noise-only files are rejected with 422.
- **Patience:** local-model calls are slow — chat ≈ 20–60 s, translation
  ≈ 30–90 s, first TTS synthesis ≈ 60–120 s (model warm-up). Every snippet
  sets generous `-m` timeouts; do not shorten them.

> [!PITFALL]
> Two `selection_criteria` mistakes bite in quick succession. **First: an
> empty criteria list 404s with "No suitable AI model found" — by design.**
> The criteria list is the contract: pick at least one `MustHave` criterion
> or the selector refuses to guess. **Second, subtler: a loose criterion
> such as `{"Domain": "GeneralPurpose"}` can match a *cloud* model**, and
> the chat call then fails with a 500 like `OPENROUTER_API_KEY is required
> for OpenRouter chat` when no OpenRouter key is configured. For a local
> sandbox, pin a local model you know is loaded, e.g.
> `{"Model": {"optionality": "MustHave", "name": "Llama3_2_1B_Q4"}}`. This
> is the same lever the Playground's Session Console exposes. Also: if
> evergreen booted while Ollama was down, its model inventory is empty —
> restart evergreen after starting Ollama.

## Companion code

```text
InayaLabs-website/docs-src/tutorials/_code/multilingual-voice-assistant/
├── voice_assistant.sh   ← the assembled driver (all checkpoints)
└── README.md            ← how to run it standalone
```

Download links: [voice_assistant.sh](_code/multilingual-voice-assistant/voice_assistant.sh) · [README.md](_code/multilingual-voice-assistant/README.md)

## Checkpoint 1 — Ask the assistant (Evergreen AI session + chat)

Create an impromptu session — every session here is created in place, no
predefined templates — then ask the garden question. Chat messages are
**multipart** (`user_message` text field), and the response is the
assistant message resource:

```bash verify=auto
KEYS=(-H "x-api-key: ${INAYA_API_KEY}" -H "x-client-app-key: ${INAYA_APP_KEY}")
sign_in() {
  curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" "${KEYS[@]}" \
    -H "Content-Type: application/json" \
    -d "{\"device_id\": \"t04-device-01\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"$1\"}" \
    | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])'
}
TOKEN=$(sign_in t04-amara)

SESS=$(curl -sf -m 180 -X POST "${EVG_URL}/v3/sessions" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}" \
  -H "Content-Type: application/json" \
  -d '{
        "system_prompt": "You are a concise community-garden assistant. Answer in one or two short sentences.",
        "temperature": 0.2,
        "required_modalities": ["Text"],
        "selection_criteria": [
          {"Model": {"optionality": "MustHave", "name": "Llama3_2_1B_Q4"}}
        ]
      }')
SESSION_ID=$(python3 -c 'import json,sys; print(json.load(sys.stdin)["ai_session_id"])' <<< "${SESS}")
echo "session ${SESSION_ID:0:16}… created"

REPLY=$(curl -sf -m 240 -X POST "${EVG_URL}/v3/sessions/${SESSION_ID}/chat-messages" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}" \
  -F "user_message=What should we plant this week? It is late September.")
ANSWER=$(python3 -c 'import json,sys
d = json.load(sys.stdin)
assert d["author"] == "Assistant", d
print(d["content"][0]["Text"])' <<< "${REPLY}")
test -n "${ANSWER}" && echo "assistant: ${ANSWER}"
echo "${ANSWER}" | grep -qi "plant\|crop\|seed\|frost\|season" \
  && echo "checkpoint 1 passed: session answered a gardening question"
```

**You should see** — the session id, a one-or-two-sentence gardening
answer (local models vary; the assertion keys on gardening vocabulary,
not exact wording), then the checkpoint line.

> [!NOTE]
> The response `content` is a **list of typed blocks** —
> `[{"Text": "…"}]` — not a bare string. Vision modalities add other block
> types; always match on the block shape you asked for (`required_modalities:
> ["Text"]` here).

## Checkpoint 2 — Translate the answer (Linga)

Linga is itself an Evergreen client: it opens a single-turn session per
translation, which is why Ollama must be healthy for this call too. Source
and target are `LanguageDescription` objects (locale, optional jargon or
vernacular registers — a farmer can get Agriculture jargon, a legal team
LegalFinancial):

```bash verify=auto
ES=$(curl -sf -m 180 -X POST "${LINGA_URL}/v3/translation/translate" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}" \
  -H "Content-Type: application/json" \
  -d "$(python3 - "$ANSWER" <<'EOF'
import json, sys
print(json.dumps({
  "source": {"locale": "en-US"},
  "target": {"locale": "es-ES"},
  "text": sys.argv[1]
}))
EOF
)")
TRANSLATED=$(python3 -c 'import json,sys
d = json.load(sys.stdin)
assert d["original_text"], d
print(d["translated_text"])' <<< "${ES}")
echo "es: ${TRANSLATED}"
python3 - "$TRANSLATED" <<'EOF'
import sys
t = sys.argv[1]
assert any(c in t.lower() for c in "áéíóúñ¿¡") or " " in t, t
print("checkpoint 2 passed: answer translated for the Spanish-speaking volunteers")
EOF
```

**You should see** — the Spanish sentence (diacritics included), then the
checkpoint line. The assertion accepts any well-formed translation output;
exact wording is model-dependent.

> [!PITFALL]
> Do not assert on *exact* translated strings. Local models are
> deterministic-ish per session (temperature 0.2 + fixed seed on the
> Evergreen side) but wording still drifts across model refreshes. Assert
> on structure: non-empty, diacritic-bearing, correct `target_language`.

## Checkpoint 3 — Register the voice (Speech voice print)

The voice print is what makes the assistant sound like Amara instead of a
default narrator. Registration takes a WAV with intelligible speech,
conforms it to 24 kHz mono, verifies it transcribes, and persists WAV +
tensor to the CDN. Make a sample with the macOS speech synthesizer (any
clear recording works), then register:

```bash verify=auto
say -o /tmp/t04-voice.aiff \
  "Hi, this is Amara. I keep the community garden notes, and I speak three languages."
afconvert -f WAVE -d LEI16@22050 /tmp/t04-voice.aiff /tmp/t04-voice.wav

VP=$(curl -sf -m 300 -X POST "${SPEECH_URL}/v3/voice-prints" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}" \
  -F "file=@/tmp/t04-voice.wav")
VOICEPRINT_PATH=$(python3 -c 'import json,sys; print(json.load(sys.stdin)["voice_print_wav_path"])' <<< "${VP}")
echo "voice print: ${VOICEPRINT_PATH}"
grep -q "^speech-service/voice-prints/" <<< "${VOICEPRINT_PATH}" \
  && echo "checkpoint 3 passed: voice print registered and persisted to the CDN"
```

**You should see** — `voice print: speech-service/voice-prints/<hash>/voice_print.wav`,
then the checkpoint line.

> [!NOTE]
> The response path is a **CDN path**, not a local file — voice prints are
> tenant-scoped objects like any other CDN asset, which is how the same
> print follows Amara across devices. Registration is one-per-user:
> re-registering overwrites (idempotent for this tutorial's re-runs).

## Checkpoint 4 — Speak the translation in Amara's voice

Now the payoff: synthesize the *Spanish* text with the *English-speaking
user's* voice print. The response is an MP3 stream (`audio/mpeg`), and the
first synthesis after boot includes model warm-up — allow real time:

```bash verify=auto
curl -sf -m 320 -o /tmp/t04-es.mp3 \
  -X POST "${SPEECH_URL}/v3/text-to-speech" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}" \
  -H "Content-Type: application/json" \
  -d "$(python3 - "$TRANSLATED" "$VOICEPRINT_PATH" <<'EOF'
import json, sys
print(json.dumps({
  "input": sys.argv[1],
  "reference_voiceprint_path": sys.argv[2]
}))
EOF
)"

python3 - <<'EOF'
data = open("/tmp/t04-es.mp3", "rb").read()
assert data[:3] == b"ID3" or data[:2] in (b"\xff\xfb", b"\xff\xf3", b"\xff\xf2"), data[:16]
print(f"audio: {len(data)} bytes, MP3 magic confirmed")
EOF
DUR=$(ffprobe -v error -show_entries format=duration -of csv=p=0 /tmp/t04-es.mp3)
python3 -c "d=float('${DUR}'); assert d > 1.0, d; print(f'checkpoint 4 passed: {d:.1f}s of Spanish speech synthesized in the registered voice')"
```

**You should see** — the byte count with MP3 magic confirmed, then
`checkpoint 4 passed: X.Xs of Spanish speech synthesized in the registered
voice`.

> [!PITFALL]
> If TTS 504s with `tts_synthesis_failed: synthesis timed out`, the
> synthesis hit `TTS_SYNTH_DEADLINE_SECS` (default 120 s) — almost always
> model warm-up on first call or the 1.7B model on a loaded machine. The
> fix is the boot env from Prerequisites (`QWEN_TTS_MODEL_DIR` → 0.6B,
> `TTS_SYNTH_DEADLINE_SECS` → 300), not retry logic in the caller. And
> note the response is MP3 even though the voice print was WAV — match on
> magic bytes, not extension.

## Checkpoint 5 — Failure injection: kill the translator both pipelines share

Kill `linga-service` and watch the blast radius: the translate call you
wrote fails **and** the TTS call you didn't know depended on Linga (input
language detection) fails too. One dependency, two consumers — exactly the
kind of coupling a composed tutorial surfaces:

```bash verify=auto
pkill -x linga-service && echo "linga-service killed"

curl -s -o /dev/null --max-time 5 \
  -X POST "${LINGA_URL}/v3/translation/translate" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}" \
  -H "Content-Type: application/json" \
  -d '{"source":{"locale":"en-US"},"target":{"locale":"ar-SA"},"text":"Water the tomatoes twice a week."}' \
  || echo "translate during outage: connection refused (fails loudly)"

CODE=$(curl -s -o /tmp/t04-tts-down.json --max-time 60 -w "%{http_code}" \
  -X POST "${SPEECH_URL}/v3/text-to-speech" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}" \
  -H "Content-Type: application/json" \
  -d "$(python3 - "$TRANSLATED" "$VOICEPRINT_PATH" <<'EOF'
import json, sys
print(json.dumps({"input": sys.argv[1], "reference_voiceprint_path": sys.argv[2]}))
EOF
)")
echo "tts during outage: HTTP ${CODE} $(head -c 80 /tmp/t04-tts-down.json 2>/dev/null)"
test "${CODE}" != "200" \
  && echo "checkpoint 5a passed: one killed service broke two pipelines — Linga is a shared dependency"

source /tmp/inaya_env.sh
export INAYA_LABS_ENVIRONMENT=local
nohup /Users/test/projects/Software/InayaLabs/target/debug/linga-service > /tmp/linga-service.log 2>&1 &
sleep 3

ES2=$(curl -sf -m 180 -X POST "${LINGA_URL}/v3/translation/translate" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}" \
  -H "Content-Type: application/json" \
  -d '{"source":{"locale":"en-US"},"target":{"locale":"ar-SA"},"text":"Water the tomatoes twice a week."}')
python3 -c 'import json,sys
d = json.load(sys.stdin)
assert d["translated_text"], d
print("checkpoint 5b passed: translator restarted — Arabic translation flowing again")' <<< "${ES2}"
```

**You should see** — `linga-service killed`, the connection-refused line, a
non-200 TTS code during the outage, both checkpoint lines, and finally an
Arabic translation proving recovery.

> [!NOTE]
> The correct resilience shape here is a **caller-side fallback**, not
> silent degradation: if language detection is unreachable, speak the text
> with a default voice rather than failing the request — or fail fast and
> let the UI say why. What a service must never do is hang: both failures
> above surface in seconds.

## Ship it

- **Production access:** point the three base URLs at the deployed hosts;
  the payloads are unchanged. Deployed Speech uses the 1.7B model with GPU
  backends — drop the `QWEN_TTS_MODEL_DIR` override.
- **Realtime voice in:** swap the typed question for
  `POST /v3/transcriptions` (audio file → text) and the loop is fully
  spoken: speech in → answer → translated → speech out in the caller's
  voice.
- **More languages:** Linga's `LanguageDescription` accepts jargon and
  vernacular registers (`{"locale":"en-US","jargon":"Agriculture"}`) — the
  garden's irrigation schedule can be translated with the right
  vocabulary.
- **Cleanup:** voice prints are CDN objects under
  `speech-service/voice-prints/` — `DELETE /v3/cdn/files/{path}` removes
  them.

## Copy agent prompt

> Follow tutorial T-04 "Build a multilingual voice assistant" at
> `/Users/test/projects/Software/InayaLabs/InayaLabs-website/docs/tutorials/multilingual-voice-assistant.html`
> end to end. Ask the garden question, translate the answer, register the
> voice print, synthesize the translated answer, then kill and restart
> linga-service. Stop if any checkpoint assertion fails and report which
> one.

### T-05 · Build an agentic inventory audit
URL: /docs/tutorials/agentic-inventory-audit.html

> **Scenario.** *Dev* closes the stockroom every evening: walk the aisles,
> eyeball the shelves, write up what needs restocking. This tutorial replaces
> the eyeball with an **agent**. You play the operator; the agent is any LLM
> client that speaks MCP (Model Context Protocol). It discovers what each
> service can do on its own — no API docs read, no SDK installed — counts the
> objects in a shelf photo, writes a dated audit entry into the searchable
> knowledge base, and answers follow-up questions from past audits. You
> compose two services: **Object Detection** (YOLO26 vision, exposed as MCP
> tools) and **RAG** (the audit log with citations).

The shape to internalize: **services are tools, and discovery is the
contract.** Every service in this workspace mounts the same MCP route —
`POST /agent/v1/mcp` (streamable HTTP, spec 2026-07-28, latest-only) — so an
agent can walk from service to service asking "what can you do?" and call
whatever it finds. The REST endpoints still exist for humans; the agent never
needs them.

**Time:** ~45 min · **Level:** advanced (JSON-RPC, base64 payloads, failure
injection)

## Architecture

```text
  you (the operator)
       │  "audit shelf A1 and log it"
       ▼
  ┌───────────── your MCP-capable agent (any LLM client) ─────────────┐
  │  1. discover:  POST /agent/v1/mcp  {method: tools/list}             │
  │     to every service it knows — no per-service code               │
  └───────┬──────────────────────────────────┬────────────────────────┘
          │ tools/call detect_objects        │ tools/call ingest_document
          ▼                                  ▼
  ┌─ Object Detection :45091 ──────┐  ┌─ RAG :40100 ──────────────────┐
  │ image_base64 ──► YOLO26 counts │  │ audit text ──► chunked, stored │
  │ {detections, count}            │  │ searchable by question         │
  └────────────────────────────────┘  └────────────────────────────────┘
          │                                  │
          └──────── audit entry ◄────────────┘
                     │
                     ▼
        tools/call search_documents ──► "what did last week's audit say?"
```

Hidden edge: **an MCP tool result is a JSON string inside `content[0].text`**
— you parse twice (JSON-RPC envelope, then the tool's own payload).
Checkpoint 2 makes that visible.

## Prerequisites

- The two services running: `object-detection-service` (**`:45091`**) and
  `rag-service` (**`:40100`**), both booted with
  `INAYA_LABS_ENVIRONMENT=sandbox` so the MCP routes accept requests without
  tenant keys or JWTs. In production the same routes require
  `x-api-key`, `x-client-app-key`, and `Authorization: Bearer <JWT>` — the
  agent sends the same headers on every call.
- Postgres running (object-detection persists detection snapshots there).
- YOLO26 ONNX models. The service's default path expects
  `~/.cache/yolo_models/yolo26x-ONNX/model.onnx`; if you only have a smaller
  variant, point the env vars at it:

  ```bash
  source /tmp/inaya_env.sh   # provides INAYA_LABS_POSTGRES_URL
  export INAYA_LABS_ENVIRONMENT=sandbox
  export OBJECT_DETECTION_GENERAL_MODEL_PATH=$HOME/.cache/yolo_models/yolo26s-ONNX/model.onnx
  export OBJECT_DETECTION_POSE_MODEL_PATH=$HOME/.cache/yolo_models/yolo26s-pose-ONNX/model.onnx
  nohup target/debug/object-detection-service > /tmp/object-detection-service.log 2>&1 &
  RAG_CHECKPOINT_PATH=/tmp/rag-stockroom-checkpoint \
    nohup target/debug/rag-service > /tmp/rag-service.log 2>&1 &
  ```

- A shelf photo. The companion code ships `stockroom.png` (shelf A1:
  cardboard boxes, shrink-wrapped bottled-water packs, a red toolbox) so the
  checkpoints are deterministic. Any photo with recognizable objects works —
  the assertions check structure, not specific inventory.
- **Patience:** the first `detect_objects` call includes ONNX model load
  (~1–5 s for `yolo26s`); subsequent calls are ~1 s.

> [!PITFALL]
> Three RAG ingest gotchas. **First: only `.pdf`, `.docx`, `.pptx`,
> `.xlsx`, and `.txt` are supported** — a `.md` file is rejected with
> `unsupported format`, so audit entries must be named `.txt` even when
> they read like markdown. **Second: the document id is derived** —
> `{collection_id}-{file_name}` — so re-ingesting the same file name
> overwrites; design file names as keys. **Third: embeddings are a
> placeholder engine in this build**, so retrieval is effectively keyword
> (BM25) search — word the audit entry and the follow-up question with
> shared vocabulary, or the search comes back empty.

## Companion code

[`_code/agentic-inventory-audit/`](./_code/agentic-inventory-audit/) —
`agentic_audit.sh` (all five checkpoints as one script, `PHOTO_PATH`
overridable) and `stockroom.png` (the audit photo).

## Checkpoint 1 — Discovery: ask every service what it can do

An agent doesn't hardcode endpoints. It walks its service list and calls
`server/discover`, then `tools/list`. The MCP route is the same on every
service in this workspace; only the port changes. Note the `Accept` header:
the endpoint answers `406` unless you accept `application/json` and/or
`text/event-stream`, and each request declares its protocol revision in
`params._meta`.

```bash verify=auto
OD_MCP="http://localhost:45091/agent/v1/mcp"
RAG_MCP="http://localhost:40100/agent/v1/mcp"
HEADERS=(-H "Content-Type: application/json" -H "Accept: application/json")
META='"_meta": {"io.modelcontextprotocol/protocolVersion": "2026-07-28"}'
# Production: add -H "x-api-key: ..." -H "x-client-app-key: ..." \
#             -H "Authorization: Bearer <JWT>" to every call below.

DISCOVER=$(curl -sf -m 15 -X POST "${OD_MCP}" "${HEADERS[@]}" \
  -d "{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"server/discover\",\"params\":{${META}}}")
echo "${DISCOVER}" | python3 -c 'import json,sys
d = json.load(sys.stdin)
result = d["result"]
info = result["_meta"]["io.modelcontextprotocol/serverInfo"]
assert info["name"] == "object-detection-service", info
assert "2026-07-28" in result["supportedVersions"], result
print("discover:", info["name"], info["version"], "speaks", ", ".join(result["supportedVersions"]))'

OD_TOOLS=$(curl -sf -m 15 -X POST "${OD_MCP}" "${HEADERS[@]}" \
  -d "{\"jsonrpc\":\"2.0\",\"id\":2,\"method\":\"tools/list\",\"params\":{${META}}}")
echo "${OD_TOOLS}" | python3 -c 'import json,sys
names = {t["name"] for t in json.load(sys.stdin)["result"]["tools"]}
assert {"detect_objects", "estimate_pose", "get_recent_detections"} <= names, names
print("object-detection tools:", sorted(names))'

RAG_TOOLS=$(curl -sf -m 15 -X POST "${RAG_MCP}" "${HEADERS[@]}" \
  -d "{\"jsonrpc\":\"2.0\",\"id\":3,\"method\":\"tools/list\",\"params\":{${META}}}")
echo "${RAG_TOOLS}" | python3 -c 'import json,sys
names = {t["name"] for t in json.load(sys.stdin)["result"]["tools"]}
assert {"ingest_document", "search_documents", "list_documents"} <= names, names
print("rag tools:", sorted(names))'
echo "checkpoint 1 passed: the agent learned both services' capabilities from server/discover + tools/list"
```

**You should see** — the discover line naming `object-detection-service`
0.1.0, both tool sets listed, and the checkpoint line. An LLM agent would
feed exactly these `description` + `inputSchema` fields into its planner.

## Checkpoint 2 — The audit: count what's on the shelf

The operator's instruction — "count the bottled water on A1-02" — becomes a
`tools/call`. The image goes up base64-encoded (the tool also accepts a
`data:` URI); `min_confidence` filters the noise floor.

```bash verify=auto
PHOTO="InayaLabs-website/docs-src/tutorials/_code/agentic-inventory-audit/stockroom.png"
[ -f "${PHOTO}" ] || PHOTO="/tmp/t05-stockroom.png"

# Build the JSON-RPC payload in Python: base64 of a 1.7 MB photo is not
# something to wrangle through shell quoting.
python3 - "${PHOTO}" <<'EOF' > /tmp/t05-detect-request.json
import base64, json, sys
img = base64.b64encode(open(sys.argv[1], "rb").read()).decode()
print(json.dumps({
    "jsonrpc": "2.0", "id": 10, "method": "tools/call",
    "params": {"name": "detect_objects",
               "arguments": {"image_base64": img, "min_confidence": 0.3}}}))
EOF

# The tool result is a JSON *string* inside content[0].text — parse twice.
RESP=$(curl -sf -m 120 -X POST "${OD_MCP}" "${HEADERS[@]}" \
  -d @/tmp/t05-detect-request.json)
python3 - "${RESP}" <<'EOF'
import json, sys
env = json.loads(sys.argv[1])
tool = json.loads(env["result"]["content"][0]["text"])   # <-- second parse
dets = tool["detections"]
assert tool["count"] == len(dets) >= 1, tool
assert any(d["object_type"] == "bottle" for d in dets), [d["object_type"] for d in dets]
for d in dets:
    assert 0.0 < d["confidence"] <= 1.0, d
    bb = d["bounding_box"]
    assert all(0.0 <= bb[k] <= 1.0 for k in ("x1", "y1", "x2", "y2")), bb
bottles = sum(1 for d in dets if d["object_type"] == "bottle")
print(f'{tool["count"]} detections in {tool["processing_time_ms"]} ms; {bottles} bottles (conf 0.30+)')
open("/tmp/t05-state.sh", "w").write(f"BOTTLE_COUNT={bottles}\n")
print("checkpoint 2 passed: shelf photo audited via tools/call detect_objects")
EOF
```

**You should see** — `N detections in <ms> ms; 7 bottles (conf 0.30+)` and
the checkpoint line, with `/tmp/t05-state.sh` carrying the count
forward. (The exact count can drift ±1 across ONNX runtimes; the assertions
only require detections, ≥1 bottle, sane confidences, and relative 0–1
boxes.)

> [!NOTE]
> YOLO is trained on COCO classes — `bottle` is one, *cardboard box* is not.
> The vision count is therefore a **partial audit by construction**: the
> model reliably sees the water packs and misses the boxes. A good audit
> entry records both what was counted and what wasn't.

## Checkpoint 3 — The log: write the audit where memory lives

The agent now writes the entry. `ingest_document` takes the file name, the
collection, and base64 content; the service chunks and indexes it. The
document id is derived — `{collection_id}-{file_name}` — so the same
nightly name overwrites, and `list_documents` is the existence check.

```bash verify=auto
source /tmp/t05-state.sh
TODAY=$(date +%F)
AUDIT_FILE="audit-${TODAY}.txt"

# Compose the audit entry: counted items from the model, uncounted noted.
python3 - "${BOTTLE_COUNT}" "${TODAY}" <<'EOF' > /tmp/t05-audit.txt
import sys
bottles, today = sys.argv[1], sys.argv[2]
print(f"""Stockroom audit {today}

Shelf A1-02 top: {bottles} bottles detected by vision audit (min confidence 0.30,
shrink-wrapped water packs on the top shelf).
Shelf A1-01 and A1 lower: cardboard boxes present but NOT counted
(cardboard box is not a COCO class the vision model reports).

Action: restock bottled water before Friday pickup.
Audited by: operator via MCP agent (object-detection-service tools/call).""")
EOF

python3 - "${AUDIT_FILE}" <<'EOF' > /tmp/t05-ingest-request.json
import base64, json, sys
content = base64.b64encode(open("/tmp/t05-audit.txt", "rb").read()).decode()
print(json.dumps({
    "jsonrpc": "2.0", "id": 20, "method": "tools/call",
    "params": {"name": "ingest_document",
               "arguments": {"collection_id": "stockroom-audits",
                             "file_name": sys.argv[1],
                             "content_base64": content}}}))
EOF

RESP3=$(curl -sf -m 60 -X POST "${RAG_MCP}" "${HEADERS[@]}" \
  -d @/tmp/t05-ingest-request.json)
python3 - "${RESP3}" "${AUDIT_FILE}" <<'EOF'
import json, sys
doc = json.loads(json.loads(sys.argv[1])["result"]["content"][0]["text"])
# rag-service namespaces document ids with the caller's tenant ("<tenant>::<collection>-<file>")
assert doc["id"].endswith("stockroom-audits-" + sys.argv[2]), doc
assert doc["chunk_count"] >= 1, doc
print("ingested:", doc["id"], "-", doc["chunk_count"], "chunk(s)")
EOF

RESP4=$(curl -sf -m 30 -X POST "${RAG_MCP}" "${HEADERS[@]}" \
  -d '{"jsonrpc":"2.0","id":21,"method":"tools/call","params":{"name":"list_documents","arguments":{"collection_id":"stockroom-audits"}}}')
python3 - "${RESP4}" "${AUDIT_FILE}" <<'EOF'
import json, sys
docs = json.loads(json.loads(sys.argv[1])["result"]["content"][0]["text"])["documents"]
names = {d["file_name"] for d in docs}
assert sys.argv[2] in names, names
print("checkpoint 3 passed: audit persisted — collection lists", len(docs), "document(s)")
EOF
```

**You should see** — the `ingested: stockroom-audits-audit-<today>.txt` line
with a chunk count, then the checkpoint line listing the collection size.
Re-running the same day overwrites the entry (same derived id) — the script
stays rerun-safe.

## Checkpoint 4 — Memory: ask yesterday's question in natural language

The payoff: next week, "what did the last audit say needs restocking?" is a
`search_documents` call, not a file hunt. Retrieved chunks carry provenance
(`file_name`, `section_hierarchy`, `score`) for citations.

```bash verify=auto
RESP5=$(curl -sf -m 60 -X POST "${RAG_MCP}" "${HEADERS[@]}" \
  -d '{"jsonrpc":"2.0","id":30,"method":"tools/call","params":{"name":"search_documents","arguments":{"collection_id":"stockroom-audits","question":"what restock action was recorded for bottled water?"}}}')
python3 - "${RESP5}" <<'EOF'
import json, sys
res = json.loads(json.loads(sys.argv[1])["result"]["content"][0]["text"])
assert res["chunks"], res
top = res["chunks"][0]
assert "restock" in top["content"].lower(), top["content"][:200]
line = next(l for l in top["content"].splitlines() if "restock" in l.lower())
print(f'top hit: {top["file_name"]} (score {top["score"]:.2f})')
print("  ", line.strip()[:90])
print("checkpoint 4 passed: natural-language question retrieved the audit entry with provenance")
EOF
```

**You should see** — `top hit: audit-<today>.txt (score …)` with the
restock line quoted and the checkpoint line. Keep the question's vocabulary
close to the entry's — this build's embeddings are placeholders, so matching
is keyword-driven (see the PITFALL above).

## Checkpoint 5 — Failure injection: blind the agent, then restore it

Kill the vision service and watch the agent's world shrink: the tool it
discovered in Checkpoint 1 is suddenly unreachable — the failure surfaces
at call time, not at planning time. Then restart and confirm recovery.

```bash verify=auto
pkill -x object-detection-service && echo "object-detection-service killed"

curl -s -m 10 -X POST "${OD_MCP}" "${HEADERS[@]}" \
  -d '{"jsonrpc":"2.0","id":40,"method":"tools/list"}' \
  || echo "discovery during outage: connection refused (fails loudly)"

source /tmp/inaya_env.sh
export INAYA_LABS_ENVIRONMENT=sandbox
export OBJECT_DETECTION_GENERAL_MODEL_PATH=$HOME/.cache/yolo_models/yolo26s-ONNX/model.onnx
export OBJECT_DETECTION_POSE_MODEL_PATH=$HOME/.cache/yolo_models/yolo26s-pose-ONNX/model.onnx
nohup /Users/test/projects/Software/InayaLabs/target/debug/object-detection-service \
  > /tmp/object-detection-service.log 2>&1 &
sleep 4

RESP6=$(curl -sf -m 120 -X POST "${OD_MCP}" "${HEADERS[@]}" \
  -d @/tmp/t05-detect-request.json)
python3 - "${RESP6}" <<'EOF'
import json, sys
tool = json.loads(json.loads(sys.argv[1])["result"]["content"][0]["text"])
assert tool["count"] >= 1, tool
print(f'checkpoint 5 passed: vision restored — {tool["count"]} detections on the same photo (model reloaded lazily on first call)')
EOF
```

**You should see** — the killed line, the connection-refused line, then the
checkpoint line with a fresh detection count. Note the restart carries the
model-path env vars: without them the service falls back to the default
`yolo26x` path and the first call 503s.

> [!NOTE]
> The resilience lesson for an agent is different from a human client's:
> the agent *re-plans*. When `tools/call` fails, a well-prompted agent falls
> back to whatever else `tools/list` offered (here: nothing — which is
> itself the finding), or tells the operator plainly. What it must not do is
> fabricate a count. This is why the audit entry names its source
> (`tools/call`) — provenance is what makes a machine-written log auditable
> by humans.

## Ship it

- **Production access:** the same MCP route on the deployed hosts, now
  with `x-api-key` / `x-client-app-key` / `Authorization: Bearer <JWT>` on
  every call; the agent sends the same headers it would to the REST API.
  No payload changes.
- **Continuous audit:** pair the service with the media-ingester pipeline
  and the `get_recent_detections` tool — snapshots land in Postgres without
  anyone photographing shelves; audits become queries.
- **Bigger brains in the loop:** wire Evergreen AI between discovery and
  action — the agent asks it to plan which tools to call, then calls them
  itself. Tutorials T-01 and T-04 show the session/chat mechanics.
- **Cleanup:** `tools/call delete_document` with the derived document id
  removes the entry; `rm -rf /tmp/rag-stockroom-checkpoint` resets the
  vector store.

## Copy agent prompt

> Follow tutorial T-05 "Build an agentic inventory audit" at
> `/Users/test/projects/Software/InayaLabs/InayaLabs-website/docs/tutorials/agentic-inventory-audit.html`
> end to end. Discover tools on object-detection (:45091) and rag (:40100)
> via tools/list, audit the stockroom photo with detect_objects, ingest the
> dated audit entry into collection `stockroom-audits`, verify it is
> retrievable with a natural-language question, then kill and restart
> object-detection-service. Stop if any checkpoint assertion fails and
> report which one.