Scenario. Fieldnote is a capture-and-publish app: users upload photos
and short clips from the field, the platform derives web-friendly
derivatives (thumbnails, trimmed clips), and a manifest of each asset's
derivatives is cached for fast reads. You will build that pipeline across
four services: CDN (durable object storage), Image Transformer and
Video Transformer (derivative workers), and Memcache (the
derivative manifest).

The shape to internalize: the CDN is the only source of truth; everything downstream is re-derivable. Transformers read from shared storage and return fresh bytes; memcache holds nothing that cannot be rebuilt.

Time: ~30 min · Level: beginner

Architecture

text

                 ┌──────────── Auth :40004 ─────────────┐
                 │  one JWT, reused by every service     │
                 └───────┬───────────────┬───────────────┘
                         │               │
        ┌────────────────▼───┐      ┌────▼───────────────────┐
        │ CDN :40040          │      │ Memcache :40011         │
        │ durable objects     │      │ derivative manifest     │
        │ (shared storage)    │      │ (in-memory, cache only) │
        └───────┬─────────────┘      └────────▲───────────────┘
                │ source_path                │ add / get / delete
   ┌────────────▼──────────────┐             │
   │ Image Transformer :40030   │  derivatives│
   │  resize · transcode        ├─────────────┘
   ├────────────────────────────┤
   │ Video Transformer :40160   │
   │  trim · transcode · mux    │
   └────────────────────────────┘

Two placement rules make this work on one host:

The CDN and both transformers share a filesystem: the CDN's

INAYA_LABS_CDN_SHARED_STORAGE_PATH is visible to the transformers, so a transformer can open an uploaded file by absolute path.

Video job outputs land in INAYA_LABS_VIDEO_OUTPUT_DIR (default: the

system temp dir). Set it to a stable path so the tutorial can find what the job produced.

Prerequisites

The local stack running: auth-service (:40004), cdn (:40040),

image-transformer-service (:40030), video-transformer-service (:40160), memcache-service (public :40011), plus ffmpeg and python3 with Pillow (both used to make test assets — the platform calls themselves are plain HTTP).

Boot the media services with the shared storage root pinned:

``bash source /tmp/inaya_env.sh export INAYA_LABS_ENVIRONMENT=local export INAYA_LABS_CDN_SHARED_STORAGE_PATH=/tmp/inaya-cdn-storage export INAYA_LABS_VIDEO_OUTPUT_DIR=/tmp/t03-video-output nohup target/debug/cdn > /tmp/cdn-service.log 2>&1 & nohup target/debug/image-transformer-service > /tmp/image-transformer-service.log 2>&1 & nohup target/debug/video-transformer-service > /tmp/video-transformer-service.log 2>&1 & ``

The tenant key pair in INAYA_API_KEY / INAYA_APP_KEY, a tenant id in

TENANT_ID (any works locally; verification used qa-tenant-t02), and a JWT (G-01 / the QA sign-in shortcut — every checkpoint below refreshes one).

Run all snippets from the repository root (/Users/test/projects/Software/InayaLabs).

PITFALL

Every service here — CDN and both transformers and memcache — requires the tenant key-pair headers (x-api-key / x-client-app-key), and every endpoint except transformer health checks also requires a valid JWT, including GET /v3/cdn/files/download?token=…. The download token authorizes the path; the JWT authorizes the caller. A token-only download attempt 401s — that is the contract, not a bug.

Companion code

text

InayaLabs-website/docs-src/tutorials/_code/media-pipeline/
├── media_pipeline.sh   ← the assembled driver (all checkpoints)
└── README.md           ← how to run it standalone

Download links: media_pipeline.sh · README.md

Checkpoint 1 — Stage an asset in the CDN and verify it round-trips

First make a test photo (any 1200×800 PNG — this stands in for a user upload), then stage it in the CDN. Upload is multipart: a path text field (namespaced per tenant) and a file bytes field:

bash

KEYS=(-H "x-api-key: ${INAYA_API_KEY}" -H "x-client-app-key: ${INAYA_APP_KEY}")
sign_in() {
  curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" "${KEYS[@]}" \
    -H "Content-Type: application/json" \
    -d "{\"device_id\": \"t03-device-01\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"$1\"}" \
    | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])'
}
TOKEN=$(sign_in t03-priya)

python3 - <<'EOF'
from PIL import Image
img = Image.new("RGB", (1200, 800), (30, 90, 160))
for x in range(0, 1200, 100):
    for y in range(0, 800, 100):
        img.putpixel((x, y), (255, 200, 40))
img.save("/tmp/t03-hero.png")
print("test asset: /tmp/t03-hero.png")
EOF

curl -sf -o /dev/null -w "upload: HTTP %{http_code}\n" \
  -X POST "${CDN_URL}/v3/cdn/files/upload?overwrite=true" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}" \
  -F "path=t03/hero.png" -F "file=@/tmp/t03-hero.png"

You should see — upload: HTTP 201. The overwrite=true query param matters on re-runs: without it, uploading to an existing path 409s.

Now mint a download token and fetch the bytes back. Note the JWT rides alongside the token:

bash

TOK=$(curl -sf -X POST "${CDN_URL}/v3/cdn/files/token" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}" \
  -H "Content-Type: application/json" \
  -d '{"path": "t03/hero.png"}')
echo "token: ${TOK:0:60}…"
DL_TOKEN=$(python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])' <<< "${TOK}")

curl -sf "http://127.0.0.1:40040/v3/cdn/files/download?token=${DL_TOKEN}" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}" \
  -o /tmp/t03-download.png
cmp /tmp/t03-hero.png /tmp/t03-download.png \
  && echo "checkpoint 1 passed: staged asset downloads byte-identical via token"

You should see — a token payload, then checkpoint 1 passed: staged asset downloads byte-identical via token.

PITFALL

Calling download?token=… without the JWT header returns 401 — {"error":"Missing Authorization token"}. Tokens scope what may be downloaded; the JWT says who is downloading.

Checkpoint 2 — Derive an image derivative

The Image Transformer takes a local absolute source_path — not a CDN URL. The uploaded file already lives on the shared filesystem under the CDN's storage root; locate it and resize:

bash

SRC=$(find "${CDN_ROOT}/files" -name "hero.png" | head -1)
test -n "${SRC}" && echo "source: ${SRC}"

curl -sf -o /tmp/t03-thumb.png \
  -X POST "${IMG_URL}/v3/images/resize" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}" \
  -H "Content-Type: application/json" \
  -d "{\"source_path\": \"${SRC}\", \"tenant_id\": \"${TENANT_ID}\", \"mode\": \"Single\", \"width\": 300, \"height\": 200}"

python3 - <<'EOF'
from PIL import Image
size = Image.open("/tmp/t03-thumb.png").size
assert size == (300, 200), size
print("checkpoint 2 passed: thumbnail derived at 300x200 (binary PNG response)")
EOF

You should see — the absolute path under the storage root, then checkpoint 2 passed: thumbnail derived at 300x200 (binary PNG response).

NOTE

The transformer response is the file (content-typed binary), not a job id — image work is synchronous and fast. Video work is not (see Checkpoint 3). Also: re-runs of this tutorial reuse the same find — the storage layout hashes tenant and namespace into directory names, so never construct these paths by hand.

Finish the loop: stage the derivative back in the CDN, next to the original:

bash

curl -sf -o /dev/null -w "derivative staged: HTTP %{http_code}\n" \
  -X POST "${CDN_URL}/v3/cdn/files/upload?overwrite=true" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}" \
  -F "path=t03/hero-thumb.png" -F "file=@/tmp/t03-thumb.png"

You should see — derivative staged: HTTP 201.

Checkpoint 3 — Derive a video clip (async job)

Video work is a 202 + job resource pattern: the service accepts the job, processes it on a worker task, and completion is announced on a notifications topic. Make a 6-second stand-in clip, stage it, and trim seconds 1–4 off it:

bash

ffmpeg -y -loglevel error \
  -f lavfi -i testsrc=duration=6:size=320x240:rate=10 -pix_fmt yuv420p \
  /tmp/t03-clip.mp4

curl -sf -o /dev/null -w "clip staged: HTTP %{http_code}\n" \
  -X POST "${CDN_URL}/v3/cdn/files/upload?overwrite=true" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}" \
  -F "path=t03/clip.mp4" -F "file=@/tmp/t03-clip.mp4"

VID_SRC=$(find "${CDN_ROOT}/files" -name "clip.mp4" | head -1)
JOB=$(curl -sf -X POST "${VID_URL}/v3/video/trim" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}" \
  -H "Content-Type: application/json" \
  -d "{\"source_path\": \"${VID_SRC}\", \"tenant_id\": \"${TENANT_ID}\", \"mode\": \"segment\", \"in_point\": \"00:00:01\", \"out_point\": \"00:00:04\"}")
JOB_ID=$(python3 -c 'import json,sys; d=json.load(sys.stdin); print(d["id"]); assert d["status"] in ("pending","running"), d' <<< "${JOB}")
echo "job ${JOB_ID:0:16}… accepted (202, status pending)"

OUT_FILE="${VID_OUT}/${TENANT_ID}/${JOB_ID}.mp4"
for i in $(seq 1 24); do
  [ -f "${OUT_FILE}" ] && { echo "job finished after ${i} poll(s)"; break; }
  sleep 5
done
test -f "${OUT_FILE}" || { echo "job never produced output"; exit 1; }

DUR=$(ffprobe -v error -show_entries format=duration -of csv=p=0 "${OUT_FILE}")
python3 -c "d=float('${DUR}'); assert 0 < d < 6, d; print(f'checkpoint 3 passed: trimmed clip is {d:.1f}s (source was 6.0s)')"

You should see — clip staged: HTTP 201, the job accepted line, job finished after N poll(s), then checkpoint 3 passed: trimmed clip is X.Xs (source was 6.0s).

PITFALL

Two video-contract sharp edges: (1) enum values deserialize snake_case — "segment", "start", "end" — and "Segment" is rejected with a 422. (2) There is no REST endpoint to poll the job: the 202 response is the last REST word, and completion arrives as a Notifications event (VideoJobPending topic, job id as sub-topic — T-01 Checkpoint 4 shows the SSE subscribe pattern). Polling the output file works everywhere, which is what the snippet does; a keyframe-aligned trim can also run slightly long, so assert a range, never an exact duration.

Checkpoint 4 — Cache the derivative manifest

Now the read-side fast path. The manifest — which derivatives exist for an asset — goes into memcache as one JSON value per asset:

bash

MANIFEST='[{"kind":"thumb","path":"t03/hero-thumb.png","width":300,"height":200},{"kind":"clip-trim","path":"video-output/'"${JOB_ID}"'.mp4","in":"00:00:01","out":"00:00:04"}]'
python3 - "$TENANT_ID" "$MANIFEST" <<'EOF' > /tmp/t03-manifest-body.json
import json, sys
print(json.dumps({"key": "t03:manifest:hero", "value": sys.argv[2], "ttl_in_seconds": 3600}))
EOF

curl -sf -o /dev/null -w "manifest cached: HTTP %{http_code}\n" \
  -X POST "${MEM_URL}/v3/mem-cache/add/${TENANT_ID}" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}" \
  -H "Content-Type: application/json" \
  --data-binary @/tmp/t03-manifest-body.json

GOT=$(curl -sf "${MEM_URL}/v3/mem-cache/get/${TENANT_ID}/t03:manifest:hero" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}")
python3 - "$GOT" <<'EOF'
import json, sys
entries = json.loads(sys.argv[1])
assert any(e["kind"] == "thumb" and e["width"] == 300 for e in entries), entries
assert any(e["kind"] == "clip-trim" for e in entries), entries
print("checkpoint 4 passed: manifest round-trips with both derivatives listed")
EOF

curl -s -o /dev/null -w "unknown key: HTTP %{http_code}\n" \
  "${MEM_URL}/v3/mem-cache/get/${TENANT_ID}/t03:manifest:nope" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}"

You should see — manifest cached: HTTP 201, the checkpoint line, and unknown key: HTTP 404 (a miss is a clean 404, not an error).

NOTE

get returns the raw stored value as the response body — not a JSON envelope — because memcache is payload-agnostic (values are AES-256 encrypted at rest and decrypted on read; CACHE_ENCRYPTION_KEY from the environment). Store strings you can parse, and parse what you get back.

Checkpoint 5 — Failure injection: the cache dies, the pipeline survives

Kill memcache and watch the failure shape: the reads fail loudly, while the source of truth (the CDN) is untouched. Then restart and observe the cache's real contract — it comes back empty:

bash

pkill -x memcache-service && echo "memcache-service killed"

curl -s -o /dev/null \
  --max-time 5 \
  "${MEM_URL}/v3/mem-cache/get/${TENANT_ID}/t03:manifest:hero" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}" \
  || echo "get during outage: connection refused (cache consumer must degrade)"

curl -sf -o /dev/null -w "cdn still serves: HTTP %{http_code}\n" \
  "http://127.0.0.1:40040/v3/cdn/files/download?token=${DL_TOKEN}" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}"
echo "checkpoint 5a passed: cache outage fails loudly; source of truth unaffected"

source /tmp/inaya_env.sh
export INAYA_LABS_ENVIRONMENT=local
nohup /Users/test/projects/Software/InayaLabs/target/debug/memcache-service > /tmp/memcache-service.log 2>&1 &
sleep 2

curl -s -o /dev/null -w "get after restart: HTTP %{http_code}\n" \
  "${MEM_URL}/v3/mem-cache/get/${TENANT_ID}/t03:manifest:hero" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}"

curl -sf -o /dev/null -w "manifest rebuilt: HTTP %{http_code}\n" \
  -X POST "${MEM_URL}/v3/mem-cache/add/${TENANT_ID}" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}" \
  -H "Content-Type: application/json" \
  --data-binary @/tmp/t03-manifest-body.json
curl -sf "${MEM_URL}/v3/mem-cache/get/${TENANT_ID}/t03:manifest:hero" \
  -H "Authorization: Bearer ${TOKEN}" "${KEYS[@]}" \
  | grep -q "hero-thumb.png" \
  && echo "checkpoint 5b passed: cache restarted empty — manifest re-derivable from CDN state, not lost"

You should see — memcache-service killed, either an HTTP error or the connection refused line, cdn still serves: HTTP 200, then get after restart: HTTP 404 (the restart wiped the in-memory cache), manifest rebuilt: HTTP 201, and the final checkpoint line.

NOTE

This is the difference between the CQRS tutorials (T-02) and this one: there, the async pipeline held the only copy of published state, so a dead pipeline meant lag. Here the cache holds derived state — the worst a restart does is force re-derivation. Design caches to be rebuildable from the source of truth, and they can die whenever they like.

Ship it

Production access: point the CDN base URL at the deployed host and

swap the local filesystem assumptions — in deployed environments the shared storage root is a mounted bucket, and transformers reach it the same way (absolute source_path).

Notifications instead of file-polling: subscribe to the

VideoJobPending topic (T-01 Checkpoint 4's SSE pattern) and let the workflow push completions instead of polling the output dir.

More derivatives: the same loop covers transcode (formats), mux

(combine clips), and upscale — and the Image Transformer's Multi resize mode derives a whole responsive set in one call.

Cleanup: DELETE /v3/cdn/files/{path} removes staged assets when

you're done.

Copy agent prompt

Follow tutorial T-03 "Build a media pipeline" at
/Users/test/projects/Software/InayaLabs/InayaLabs-website/docs/tutorials/media-pipeline.html
end to end. Stage both test assets, derive the image thumbnail and the
trimmed clip, cache the manifest, then kill and restart memcache. Stop if
any checkpoint assertion fails and report which one.