Inaya Labs | Docs — Cloning a voice and speaking

Journey. As a developer, I want my app to let a user register their
voice once, then speak any text in that voice — with a consent-verified
option for trusted, anti-spoof enrollment — and receive audio my client
can play or stream.

The Speech service (:40161, prefix /v3) runs a local Qwen3-TTS voice-cloning model (MLX on macOS) — no cloud TTS API key, no per-call vendor cost. Voice cloning is a two-resource dance:

A VoicePrint — a 24 kHz mono WAV of the user's voice, uploaded as

multipart audio, Whisper-checked for intelligible speech, stored in the CDN. Registration returns a path, voice_print_wav_path, which you pass back as reference_voiceprint_path on every synthesis call.

Speech synthesis — POST /v3/text-to-speech clones the reference

voice and returns binary audio (MP3 by default), or an SSE stream of base64 PCM chunks when stream=true.

Everything is snake_case; field names are long and literal (reference_voiceprint_path, voice_print_wav_path).

Prerequisites

The Speech service boots only when its local models are present:

Qwen3-TTS (QWEN_TTS_MODEL_DIR, default ~/.InayaLabs/models/hf/Qwen3-TTS-12Hz-1.7B-Base) and Whisper-tiny (WHISPER_TINY_MODEL_DIR). ffmpeg must be on PATH.

CDN (voice-print storage) must be running; the verified enrollment

flow additionally needs Memcache (script TTL storage) and Evergreen AI (script generation). The unverified flow needs only the CDN.

Auth is the standard pair: x-api-key + x-client-app-key and a user

JWT, which the service forwards to the CDN when downloading your voice print. On macOS you can fabricate a test recording with say -o sample.wav --data-format=LEI16@24000 "…".

One voice print per user: re-registering overwrites the previous one.

There are no list/get/delete endpoints; the path embeds sha256(JWT subject), so it is deterministic per user.

Concepts

Two enrollment flows.

Unverified (POST /v3/voice-prints): one multipart upload, single step, right for low-trust or prototyping scenarios. Verified (two steps: POST /v3/verified-voice-prints/verifiable-paragraphs → read the returned script aloud → POST /v3/verified-voice-prints/{registration_id} with the recording): the service LLM-generates a fresh consent script, Whisper-transcribes your recording, and requires ≥ 0.90 similarity — proving the speaker read this script now. Registration IDs are single-use and expire after 10 minutes.

Synthesis responses are binary, not JSON. Non-streaming TTS returns

raw audio bytes (audio/mpeg by default); save with -o. Set output_format to wav (or any ffmpeg format) to change the container.

stream=true switches to SSE: events {"type":"speech.audio.delta","delta":"<base64 PCM>"}

(16-bit little-endian, 24 kHz mono) arrive as synthesis progresses; the HTTP stream simply ends at natural completion. A speech.audio.done event is the failure terminator — when you see it, synthesis failed (check service logs); it is not sent on success.

Cold-start latency is real. The first synthesis after boot compiles

the MLX graph and can exceed the internal 120 s synthesis deadline (surfacing as 504 tts_synthesis_failed); a retry after warmup completes in ~20–45 s. Plan client timeouts accordingly.

Optional steering: style_prompt ({"kind":"preset","preset":"cheerful"}

or {"kind":"custom","text":"…"}), profanity scrub (clean + locale), and cross-lingual translation via Linga (target). speed is reserved and ignored.

Step 1 — Get a user JWT

bash

USER_TOKEN=$(curl -sf -X POST "${AUTH_URL}/v3/auth/qa/login" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -d "{\"device_id\": \"g61-demo\", \"locale\": \"en-US\", \"tenant_id\": \"${TENANT_ID}\", \"username\": \"g61-speaker\"}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])')
test -n "${USER_TOKEN}" && echo "JWT acquired (${#USER_TOKEN} chars)"

Expected — a 24 h PS512 JWT. (qa/login requires device_id of at least 8 chars; it provisions the user on first use via the Users services.)

Step 2 — Record a voice sample and register (unverified flow)

Fabricate a clean test recording (macOS say, or use any microphone WAV — the service conforms it to 24 kHz mono and rejects audio Whisper can't transcribe with 422):

bash

export WORKDIR=$(mktemp -d /tmp/g61-voice.XXXXXX)
say -o "${WORKDIR}/voice.wav" --data-format=LEI16@24000 \
  "Hello, this is my voice reference. I am reading this sentence clearly so the system can learn how I sound."
test -s "${WORKDIR}/voice.wav" && echo "sample recorded: $(stat -f%z ${WORKDIR}/voice.wav) bytes"

VP_JSON=$(curl -s -X POST "${SPEECH_URL}/v3/voice-prints" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}" \
  -F "file=@${WORKDIR}/voice.wav")
echo "${VP_JSON}"
VOICEPRINT_PATH=$(echo "${VP_JSON}" | python3 -c 'import json,sys; print(json.load(sys.stdin)["voice_print_wav_path"])')
echo "${VOICEPRINT_PATH}" | grep -q "speech-service/voice-prints/" && echo "voice print registered: ${VOICEPRINT_PATH}"

Expected response — 200: {"voice_print_wav_path":"speech-service/voice-prints/<sha256(user sub)>/voice_print.wav"}. The multipart field must be exactly file; extra fields → 400. This call takes ~10 s (Whisper intelligibility check + CDN upload).

Step 3 — Speak with the cloned voice

bash

curl -s -o "${WORKDIR}/spoken.mp3" -w 'HTTP %{http_code} | %{content_type} | %{size_download} bytes\n' \
  -X POST "${SPEECH_URL}/v3/text-to-speech" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}" \
  -d "{\"input\": \"Hello! This sentence is spoken with my cloned voice.\", \"reference_voiceprint_path\": \"${VOICEPRINT_PATH}\", \"locale\": \"en\"}"
file "${WORKDIR}/spoken.mp3"
ffprobe -v error -show_entries format=format_name,duration -of csv=p=0 "${WORKDIR}/spoken.mp3"
file "${WORKDIR}/spoken.mp3" | grep -qi "Audio file" && echo "synthesis verified"

Expected — 200 with an MP3 body (ID3-tagged MPEG audio, ~24 kHz mono). locale (en, ja-JP, …) must match the language of input. Change the container with output_format: "wav". This is the slow call: ~20–45 s warm; first-call-after-boot can exceed the internal deadline and return 504 tts_synthesis_failed — retry rather than treat as fatal.

Step 4 — Stream the audio as SSE

For play-as-it-generates clients, set stream=true and consume the SSE events; each delta is base64 16-bit-LE PCM at 24 kHz mono:

bash

SSE_OUT=$(mktemp /tmp/g61-tts.XXXXXX.txt)
curl -s -N -X POST "${SPEECH_URL}/v3/text-to-speech" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}" \
  -d "{\"input\": \"Streaming check with the cloned voice.\", \"reference_voiceprint_path\": \"${VOICEPRINT_PATH}\", \"locale\": \"en\", \"stream\": true}" \
  > "${SSE_OUT}"
DELTAS=$(grep -c 'speech.audio.delta' "${SSE_OUT}")
echo "delta events: ${DELTAS}"
[ "${DELTAS}" -gt 0 ] || { echo "no audio deltas streamed"; exit 1; }
python3 - "${SSE_OUT}" <<'PY'
import base64, sys
pcm = b""
for line in open(sys.argv[1]):
    if line.startswith("data:") and "speech.audio.delta" in line:
        import json
        pcm += base64.b64decode(json.loads(line[5:])["delta"])
assert len(pcm) > 8000, len(pcm)  # > ~0.17 s of 16-bit 24 kHz mono
open(sys.argv[1] + ".pcm", "wb").write(pcm)
print(f"decoded PCM: {len(pcm)} bytes")
PY

Expected — several data: {"type":"speech.audio.delta","delta":"…"} events (~10 s total); the HTTP response ends at natural completion (no terminal event on success). A speech.audio.done event means synthesis failed mid-stream.

For trusted enrollment, use the two-step verified flow. The service generates a fresh script via Evergreen AI; you have 10 minutes to record it:

bash

BEGIN_JSON=$(curl -s -X POST "${SPEECH_URL}/v3/verified-voice-prints/verifiable-paragraphs" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}" \
  -d '{"locale": "en-US"}')
echo "${BEGIN_JSON}"
REG_ID=$(echo "${BEGIN_JSON}" | python3 -c 'import json,sys; print(json.load(sys.stdin)["registration_id"])')
SCRIPT=$(echo "${BEGIN_JSON}" | python3 -c 'import json,sys; print(json.load(sys.stdin)["script"])')
test -n "${REG_ID}" && echo "registration id: ${REG_ID} (TTL 600 s)"

Expected response — 200 with registration_id and a short LLM-generated script. 503 here means Evergreen AI is unreachable.

Record the script (a human reads it; say stands in for CI), then finalize:

bash

say -o "${WORKDIR}/verified.wav" --data-format=LEI16@24000 "${SCRIPT}"
VERIFIED_JSON=$(curl -s -X POST "${SPEECH_URL}/v3/verified-voice-prints/${REG_ID}" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}" \
  -F "file=@${WORKDIR}/verified.wav")
echo "${VERIFIED_JSON}"
echo "${VERIFIED_JSON}" | python3 -c '
import json, sys
r = json.load(sys.stdin)
assert r["voice_print_wav_path"].startswith("speech-service/voice-prints/"), r
print("verified voice print:", r["voice_print_wav_path"])
'

Expected — 200 with the same voice_print_wav_path shape (the verified print overwrites the unverified one — one print per user). Failures: 404 (unknown/expired registration id), 422 (recording not intelligible or transcript similarity < 0.90 vs. the script).

Step 6 — Error contract

bash

# 400 — unknown voice print path on synthesis (plain text body)
CODE=$(curl -s -o /tmp/g61-err1.txt -w '%{http_code}' -X POST "${SPEECH_URL}/v3/text-to-speech" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -H "Authorization: Bearer ${USER_TOKEN}" \
  -d '{"input": "test", "reference_voiceprint_path": "speech-service/voice-prints/nobody/voice_print.wav", "locale": "en"}')
[ "${CODE}" = "400" ] && grep -q "Voice print not found" /tmp/g61-err1.txt \
  && echo "400 plain-text error ok: $(cat /tmp/g61-err1.txt)"

# 401 — missing JWT on registration (JSON body)
CODE=$(curl -s -o /tmp/g61-err2.txt -w '%{http_code}' -X POST "${SPEECH_URL}/v3/voice-prints" \
  -H "x-api-key: ${INAYA_API_KEY}" \
  -H "x-client-app-key: ${INAYA_APP_KEY}" \
  -F "file=@${WORKDIR}/voice.wav")
[ "${CODE}" = "401" ] && echo "401 auth error ok: $(cat /tmp/g61-err2.txt)"

Expected — 400 with a plain-text body for a missing voice print, and 401 JSON when the JWT is absent. (Some validation failures elsewhere in this service return RFC-7807 Problem JSON — parse by status first, then sniff the body.)

Putting it together

bash

KEYS=(-H "x-api-key: $INAYA_API_KEY" -H "x-client-app-key: $INAYA_APP_KEY")

# 1. Register (unverified)
VP=$(curl -sf -X POST "$SPEECH_URL/v3/voice-prints" -H "Authorization: Bearer $USER_TOKEN" \
  "${KEYS[@]}" -F "file=@voice.wav" | python3 -c 'import json,sys; print(json.load(sys.stdin)["voice_print_wav_path"])')

# 2. Speak (binary MP3 out)
curl -sf -o spoken.mp3 -X POST "$SPEECH_URL/v3/text-to-speech" \
  -H "Authorization: Bearer $USER_TOKEN" "${KEYS[@]}" -H "Content-Type: application/json" \
  -d "{\"input\": \"Status report ready.\", \"reference_voiceprint_path\": \"$VP\", \"locale\": \"en\"}"

# 3. Or stream (SSE base64 PCM deltas)
curl -sN -X POST "$SPEECH_URL/v3/text-to-speech" \
  -H "Authorization: Bearer $USER_TOKEN" "${KEYS[@]}" -H "Content-Type: application/json" \
  -d "{\"input\": \"Status report ready.\", \"reference_voiceprint_path\": \"$VP\", \"locale\": \"en\", \"stream\": true}"

(SPEECH_URL is http://127.0.0.1:40161 on the local stack; AUTH_URL is http://127.0.0.1:40004; TENANT_ID is your tenant. Voice-print registration also needs the CDN on :40040; the verified flow adds Memcache on :40011 and Evergreen AI on :40071.)

Try it live

Open the Playground → Translation & Voice Cloning — the Speech console (VoicePrint + Speak modes) exercises the same register → synthesize cycle against the live service.

Copy agent prompt

Agent prompt — copy and hand to your agent

You are working with the Inaya Labs Speech service at SPEECH_URL (:40161,
prefix /v3), which runs a LOCAL Qwen3-TTS voice-cloning model (no cloud
TTS key). The tenant key pair is in INAYA_API_KEY / INAYA_APP_KEY and
TENANT_ID; AUTH_URL (:40004) mints user JWTs via POST /v3/auth/qa/login
{"device_id" (min 8 chars), "locale": "en-US", "tenant_id", "username"}.
EVERY call needs the key-pair headers AND a user JWT (forwarded to the CDN
for voice-print download). Steps: (1) Fabricate a test voice recording with
macOS `say -o voice.wav --data-format=LEI16@24000 "<one clear sentence>".
(2) Register an UNVERIFIED voice print: POST SPEECH /v3/voice-prints with
multipart field `file=@voice.wav` — assert 200 and capture
voice_print_wav_path (a CDN path like speech-service/voice-prints/<hash>/
voice_print.wav). One print per user; re-registering overwrites. (3)
Synthesize: POST SPEECH /v3/text-to-speech {"input", "reference_voiceprint_path",
"locale": "en"} — the RESPONSE BODY IS BINARY MP3 (save with -o; ~20-45 s
warm; first-call-after-boot may 504 tts_synthesis_failed — retry). Assert
with `file` that it is MPEG audio. (4) Stream: repeat with "stream": true —
parse SSE `data: {"type":"speech.audio.delta","delta":"<base64 PCM>"}`
events (16-bit LE, 24 kHz mono), base64-decode and assert > 8 KB of PCM;
stream ends silently at completion (speech.audio.done is the FAILURE
terminator). (5) VERIFIED enrollment: POST /v3/verified-voice-prints/
verifiable-paragraphs {"locale":"en-US"} → capture registration_id (TTL
600 s) and script; record the script and POST it to /v3/verified-voice-
prints/{registration_id} with -F file=@recording.wav — assert 200 (422 =
transcript similarity < 0.90). (6) Errors: unknown voiceprint path on TTS
→ 400 PLAIN TEXT "Voice print not found"; missing JWT on registration →
401 JSON. Report status codes, byte counts, and PCM sizes per step; never
log the JWT.

API reference

Speech Service — Swagger UI

Next journeys

Translating text with Linga (G-60) — the translation engine behind cross-lingual speech.
Transforming audio, images & video (G-50) — trim/transcode the audio you just produced.
Uploading files (G-02) — how files reach the CDN that stores voice prints.

Prerequisites

Concepts

Step 1 — Get a user JWT

Step 2 — Record a voice sample and register (unverified flow)

Step 3 — Speak with the cloned voice

Step 4 — Stream the audio as SSE

Step 5 — Verified enrollment (consent script)

Step 6 — Error contract

Putting it together

Try it live

Copy agent prompt

API reference

Next journeys