/api/voice (for example, POST /api/voice/speak). All requests authenticate with your PeachDesk API key in the X-API-Key header (an interactive session token in Authorization: Bearer also works — see Authentication).
The speech-to-text, language, and text-to-speech models that power conversation all run on PeachDesk’s GPU infrastructure. You select an engine and model sizes from a dropdown in Voice Studio (or by name in the API) — there is nothing to download or install locally.
Authentication
Send your PeachDesk API key in theX-API-Key header on every HTTP request (an
interactive session token in Authorization: Bearer also works — see
Authentication).
Live conversation (WebSocket)
A full-duplex, turn-based (“push-to-talk”) voice agent. The client records a user utterance, sends it as a binary frame, and plays back the streamed spoken reply. Typed input and conversation reset are also supported, and a new utterance automatically interrupts (barges in on) any reply still being spoken.WS /api/voice/ws/converse
Open a WebSocket and exchange JSON control frames plus binary audio frames for an ongoing conversation.
Connection lifecycle
1
Open the socket
Connect to
wss://<your-peachdesk-host>/api/voice/ws/converse?api_key=bibha_your_api_key (or ?token=<session_token>). The credential must be in the query string — a browser WebSocket cannot send headers.2
Send the init frame
The first frame must be a JSON text frame that selects the voice profile and the models to use for the session.
3
Wait for ready
The server replies with a
ready event once the voice and models are loaded on PeachDesk’s servers.4
Exchange turns
Per turn, send either a binary audio frame (a recorded utterance) or a
text control frame. The server streams back the transcript, the reply text, and WAV audio chunks, then a turn_end event.Init frame (client → server)
The first text frame after connecting selects the profile and model stack for the session.string
required
ID of the voice profile the agent should speak with. The profile may be a Clone from audio voice or a Built-in voice. If the profile is a clone, the Voice Engine automatically resolves a cloning-capable engine.
string
default:"vaani"
TTS engine to speak with. One of
vaani (VAANI), vaani-english (VAANI-ENGLISH), vaani-multilingual (VAANI-MULTILINGUAL), or vaani-indic (VAANI-INDIC). Defaults to the profile’s preferred engine, falling back to vaani. All engines run on PeachDesk infrastructure.string
Legacy field — no served engine exposes multiple sizes. Omit it.
string
default:"lipi"
Speech-to-text model used to transcribe the user’s utterances. Leave it at
lipi: live turns are handled by the low-latency LIPI-Fast runtime automatically when its weights are present, falling back to LIPI otherwise. The family’s other models (lipi-bharat, lipi-live) decode a finished recording in one pass, so they belong on the capture routes, not on a live call.string
default:"BRAHMA-30B-FP8"
The language model that generates the agent’s replies.
BRAHMA-30B-FP8 (BRAHMA-30B) is the only served value.string
default:"en"
Conversation language as an ISO code (for example
en, es, fr, de, ja).integer
default:"1"
Beam size for speech-to-text decoding.
1 (greedy) gives the lowest latency and is recommended for live conversation.Per-turn frames (client → server)
Server → client events
object
Sent once after init. Includes
profile_id, name, engine, stt_model, and llm_model for the session.object
What the agent heard.
{"type": "user_transcript", "text": "..."}object
The full reply text.
{"type": "agent_transcript", "text": "..."}binary
WAV audio chunks of the spoken reply, streamed sentence-by-sentence. Play them in order as they arrive.
object
Marks the end of a reply.
{"type": "turn_end"}object
Non-fatal notices, e.g.
{"type": "info", "message": "No speech detected."} or a reset confirmation.object
An error occurred.
{"type": "error", "message": "..."}Example (browser)
Barge-in is handled for you: sending a new utterance (or an
interrupt frame) cancels the in-progress reply so the agent stops talking and the new turn is handled immediately. The interrupted reply is not added to conversation memory.Speak on demand
A simple REST endpoint that asks an agent to speak a line of text in a chosen voice profile. It mirrors the conversation pipeline and the standard generation flow, so the response and polling behaviour matchPOST /api/voice/generate.
POST /api/voice/speak
Generate spoken audio for a piece of text in a voice profile.
string
required
The text to speak. 1–10,000 characters.
string
Voice profile name or id. If omitted, the Voice Engine falls back to a per-client binding and then your default profile. A
400 is returned if no profile can be resolved, or 404 if a named profile is not found.string
TTS engine to use. One of
vaani (VAANI), vaani-english (VAANI-ENGLISH), vaani-multilingual (VAANI-MULTILINGUAL), or vaani-indic (VAANI-INDIC). Defaults to the profile or client-binding preference.boolean
When
true and the profile has a personality prompt, the text is rewritten in-character before it is spoken. When omitted, the per-client binding’s default decides.string
default:"en"
Language ISO code for the spoken output (for example
en, es, ja).Response
Returns aGenerationResponse with status: "generating" and an id you poll at GET /api/voice/generate/{id}/status until the audio is ready.
string
required
Generation ID. Poll its status to retrieve the finished audio.
string
required
ID of the resolved voice profile.
string
required
The text that is being spoken.
string
required
Language of the output.
string
Generation status, e.g.
generating, then completed once ready.string
Engine used for synthesis.
string
Path to the generated audio once
status is completed. Null while generating.number
Length of the generated audio in seconds (when available).
string
required
ISO 8601 timestamp when the generation was created.
Example
Speak events (Server-Sent Events)
A live stream of speaking-state events. Voice Studio uses it to show when an agent-initiated speak (fromPOST /api/voice/speak) is playing.
GET /api/voice/events/speak
Subscribe to a Server-Sent Events stream of speak-start and speak-end events.
Events
Example
This is a read-only event stream — it does not trigger speech. Pair it with
POST /api/voice/speak to drive UI that reflects when an agent is talking.
