Skip to main content
WSS
Preview — invitation only. Duplex API access is currently limited to selected customers. Public documentation does not grant service access.
Use this endpoint when you want Fish to provide turn detection. Read the Duplex API guide for the architecture, starter application, and conversation flow. Select a voice with session.start.voice.reference_id. See Voice presets for Clara and the other ready-to-use reference IDs, or supply a custom Fish Voice Library ID. Use the ID, not the voice’s display name. Every message is a binary WebSocket frame containing a MessagePack map. Audio payloads use MessagePack binary data, not base64 strings.
bearerAuth
type:http

Pass your Fish Audio API key on the server-to-server WebSocket upgrade request: Authorization: Bearer YOUR_API_KEY

session.start
type:object

Sent once as the very first frame within 5 s of connecting (or within 1 s when server capacity is full). Configures voice, audio format, and eager turn options for the entire call.

audio
type:object

Continuous real-time caller audio stream (20 ms to 100 ms per frame), silence included. Acts as the session clock; stopping for over 30 s closes the call (4408 idle_timeout) unless listen.pause is active.

reply.text
type:object

Streams text chunks from your LLM for the agent to speak. Starting a new reply_id preempts any currently speaking reply.

reply.cancel
type:object

Cancels an in-progress or held speculative reply (for example, when turn.resumed arrives).

listen.pause
type:object

Mutes caller input processing (during IVR keypad entry, call transfer, or hold) and extends idle timeout from 30 s to 300 s.

listen.resume
type:object

Resumes caller audio listening and restores the 30 s idle timeout.

session.started
type:object

Confirms the session is live, returning active limits, audio rates, and LLM prompt text_rules. Begin streaming real-time caller audio upon receipt.

audio
type:object

Continuous pcm_s16le mono agent audio stream at output_sample_rate (silence included). Buffer ~240–300 ms on playback.

turn.eager
type:object

Speculative end-of-turn signal emitted as soon as the caller pauses. Start your LLM stream immediately and send reply.text with "for": eager_id (gate mutating database/order writes until turn.committed).

turn.resumed
type:object

Caller continued speaking after turn.eager. Cancel your in-flight speculative LLM stream and send reply.cancel.

turn.committed
type:object

Authoritative finalized caller turn. If approved_eager_id matches your active speculative run, Fish Audio immediately releases the pre-synthesized audio buffer (lead_ms reports latency saved) and your server can commit any gated write tools.

turn.dropped
type:object

Sound stopped agent speech or triggered VAD without becoming a valid caller turn.

turn.backchannel
type:object

Short caller acknowledgement ("mm-hmm", "okay") while the agent holds the floor.

turn.interim
type:object

Live streaming ASR partial transcript for UI captions (turns.interim: true).

reply.started
type:object

Emitted when the first audible speech sample of reply_id enters the output audio stream.

reply.progress
type:object

Periodic spoken character progress along reply_id.

reply.interrupted
type:object

Emitted up to twice when a reply is cut short (final: false at cut onset for immediate client speaker fade-out; final: true once agent audio settles).

reply.done
type:object

Single authoritative receipt concluding every reply. Always append heard_text (never unplayed unheard_text) to your LLM conversation history.

agent.speech
type:object

Autonomous conversational backchannel or natural vocalization produced by the voice model outside explicit replies.

session.ending
type:object

Warning emitted 120 s and 30 s before limits.max_session_s.

error
type:object

Recoverable or fatal protocol/subsystem error notification.