The Realtime Conversation resource powers the Talk tab in PeachDesk Voice Studio. It lets an application hold a spoken, back-and-forth conversation with a voice agent, ask the agent to speak a line of text on demand, and listen for live speaking-state events to drive UI. Every endpoint is proxied by PeachDesk under the base path /api/voice (for example, POST /api/voice/speak). All requests authenticate with your PeachDesk API key in the X-API-Key header (an interactive session token in Authorization: Bearer also works — see Authentication).
The speech-to-text, language, and text-to-speech models that power conversation all run on PeachDesk’s GPU infrastructure. You select an engine and model sizes from a dropdown in Voice Studio (or by name in the API) — there is nothing to download or install locally.

Authentication

Send your PeachDesk API key in the X-API-Key header on every HTTP request (an interactive session token in Authorization: Bearer also works — see Authentication).
The WebSocket endpoint is the exception: a browser WebSocket cannot set request headers, so the credential travels in the query string instead — ?api_key=bibha_your_api_key or ?token=<session_token>. Everything else about the credential is unchanged; only the transport differs.

Live conversation (WebSocket)

A full-duplex, turn-based (“push-to-talk”) voice agent. The client records a user utterance, sends it as a binary frame, and plays back the streamed spoken reply. Typed input and conversation reset are also supported, and a new utterance automatically interrupts (barges in on) any reply still being spoken.

WS /api/voice/ws/converse

Open a WebSocket and exchange JSON control frames plus binary audio frames for an ongoing conversation.

Connection lifecycle

1

Open the socket

Connect to wss://<your-peachdesk-host>/api/voice/ws/converse?api_key=bibha_your_api_key (or ?token=<session_token>). The credential must be in the query string — a browser WebSocket cannot send headers.
2

Send the init frame

The first frame must be a JSON text frame that selects the voice profile and the models to use for the session.
3

Wait for ready

The server replies with a ready event once the voice and models are loaded on PeachDesk’s servers.
4

Exchange turns

Per turn, send either a binary audio frame (a recorded utterance) or a text control frame. The server streams back the transcript, the reply text, and WAV audio chunks, then a turn_end event.

Init frame (client → server)

The first text frame after connecting selects the profile and model stack for the session.
string
required
ID of the voice profile the agent should speak with. The profile may be a Clone from audio voice or a Built-in voice. If the profile is a clone, the Voice Engine automatically resolves a cloning-capable engine.
string
default:"vaani"
TTS engine to speak with. One of vaani (VAANI), vaani-english (VAANI-ENGLISH), vaani-multilingual (VAANI-MULTILINGUAL), or vaani-indic (VAANI-INDIC). Defaults to the profile’s preferred engine, falling back to vaani. All engines run on PeachDesk infrastructure.
string
Legacy field — no served engine exposes multiple sizes. Omit it.
string
default:"lipi"
Speech-to-text model used to transcribe the user’s utterances. Leave it at lipi: live turns are handled by the low-latency LIPI-Fast runtime automatically when its weights are present, falling back to LIPI otherwise. The family’s other models (lipi-bharat, lipi-live) decode a finished recording in one pass, so they belong on the capture routes, not on a live call.
string
default:"BRAHMA-30B-FP8"
The language model that generates the agent’s replies. BRAHMA-30B-FP8 (BRAHMA-30B) is the only served value.
string
default:"en"
Conversation language as an ISO code (for example en, es, fr, de, ja).
integer
default:"1"
Beam size for speech-to-text decoding. 1 (greedy) gives the lowest latency and is recommended for live conversation.
The defaults are tuned so speech-to-text, the language model, and text-to-speech all fit in memory at once — vaani is the lightest served voice engine (~1.5 GB), which is what leaves room for LIPI alongside it. Selecting one of the full-size engines (vaani-multilingual, vaani-english, vaani-indic) costs a model swap on the first turn.

Per-turn frames (client → server)

Server → client events

object
Sent once after init. Includes profile_id, name, engine, stt_model, and llm_model for the session.
object
What the agent heard. {"type": "user_transcript", "text": "..."}
object
The full reply text. {"type": "agent_transcript", "text": "..."}
binary
WAV audio chunks of the spoken reply, streamed sentence-by-sentence. Play them in order as they arrive.
object
Marks the end of a reply. {"type": "turn_end"}
object
Non-fatal notices, e.g. {"type": "info", "message": "No speech detected."} or a reset confirmation.
object
An error occurred. {"type": "error", "message": "..."}

Example (browser)

Barge-in is handled for you: sending a new utterance (or an interrupt frame) cancels the in-progress reply so the agent stops talking and the new turn is handled immediately. The interrupted reply is not added to conversation memory.

Speak on demand

A simple REST endpoint that asks an agent to speak a line of text in a chosen voice profile. It mirrors the conversation pipeline and the standard generation flow, so the response and polling behaviour match POST /api/voice/generate.

POST /api/voice/speak

Generate spoken audio for a piece of text in a voice profile.
string
required
The text to speak. 1–10,000 characters.
string
Voice profile name or id. If omitted, the Voice Engine falls back to a per-client binding and then your default profile. A 400 is returned if no profile can be resolved, or 404 if a named profile is not found.
string
TTS engine to use. One of vaani (VAANI), vaani-english (VAANI-ENGLISH), vaani-multilingual (VAANI-MULTILINGUAL), or vaani-indic (VAANI-INDIC). Defaults to the profile or client-binding preference.
boolean
When true and the profile has a personality prompt, the text is rewritten in-character before it is spoken. When omitted, the per-client binding’s default decides.
string
default:"en"
Language ISO code for the spoken output (for example en, es, ja).

Response

Returns a GenerationResponse with status: "generating" and an id you poll at GET /api/voice/generate/{id}/status until the audio is ready.
string
required
Generation ID. Poll its status to retrieve the finished audio.
string
required
ID of the resolved voice profile.
string
required
The text that is being spoken.
string
required
Language of the output.
string
Generation status, e.g. generating, then completed once ready.
string
Engine used for synthesis.
string
Path to the generated audio once status is completed. Null while generating.
number
Length of the generated audio in seconds (when available).
string
required
ISO 8601 timestamp when the generation was created.

Example

After the call returns, poll GET /api/voice/generate/{id}/status with the id from the response until status becomes completed, then fetch the audio. This matches the standard generation flow used by the Generation tab.

Speak events (Server-Sent Events)

A live stream of speaking-state events. Voice Studio uses it to show when an agent-initiated speak (from POST /api/voice/speak) is playing.

GET /api/voice/events/speak

Subscribe to a Server-Sent Events stream of speak-start and speak-end events.

Events

Example

This is a read-only event stream — it does not trigger speech. Pair it with POST /api/voice/speak to drive UI that reflects when an agent is talking.