{
"event": "start",
"request": {
"text": "",
"format": "mp3",
"chunk_length": 300,
"reference_id": "9a9cf47702da476aa4629e2506d4a857",
"latency": "normal"
}
}{
"event": "text",
"text": "Hello, this is streaming text. "
}{
"event": "flush"
}{
"event": "stop"
}{
"event": "audio",
"audio": "<binary audio data>",
"content": "Hello there.",
"alignment": {
"segments": [
{
"text": "Hello",
"start": 0,
"end": 0.42
},
{
"text": "there.",
"start": 0.42,
"end": 0.86
}
],
"audio_duration": 0.86
},
"chunk_seq": 0,
"chunk_audio_offset_sec": 0,
"time": 920
}{
"event": "finish",
"reason": "stop",
"time": 5460
}{
"event": "error",
"error": "Concurrency limit exceeded",
"max_concurrency": 8
}WebSocket TTS with Timestamps
Stream text and receive speech with word-level timestamps over WebSocket
wss://api.fish.audio/v1/tts/live/with-timestamp with a WebSocket
client. The endpoint requires a WebSocket upgrade and returns MessagePack
binary frames. A normal HTTP GET request does not start synthesis.Before you start
You need a Fish Audio API key and a WebSocket client that supports custom authentication headers. To run the Python example on this page, install its dependencies:pip install ormsgpack websockets
<token> in the example with your API key and set reference_id to your
voice model ID. The start.request object accepts the same parameters as the
Text to Speech API.
Send text and receive audio concurrently, then keep reading after sending stop
until the server sends finish.
Preserve the final timestamps
Process alignment metadata even when anaudio event contains empty audio bytes:
a trailing event can carry a final alignment correction. Store each non-null
alignment by chunk_seq, replacing its previous snapshot. A null alignment
does not erase a snapshot you already received.
Add chunk_audio_offset_sec to each segment’s start and end to place it on
the session’s audio timeline. The optional time field measures elapsed server
session time in milliseconds; use the segment timestamps for captions.
After finish, you can send another start on the same socket. Reset your audio
buffer and stored alignment snapshots for the new session.
Related endpoints
- Text to Speech Stream with Timestamps: send the full text in one HTTP request and receive audio and timestamps over SSE.
- WebSocket TTS Streaming: stream text and audio without timestamps.
{
"event": "start",
"request": {
"text": "",
"format": "mp3",
"chunk_length": 300,
"reference_id": "9a9cf47702da476aa4629e2506d4a857",
"latency": "normal"
}
}{
"event": "text",
"text": "Hello, this is streaming text. "
}{
"event": "flush"
}{
"event": "stop"
}{
"event": "audio",
"audio": "<binary audio data>",
"content": "Hello there.",
"alignment": {
"segments": [
{
"text": "Hello",
"start": 0,
"end": 0.42
},
{
"text": "there.",
"start": 0.42,
"end": 0.86
}
],
"audio_duration": 0.86
},
"chunk_seq": 0,
"chunk_audio_offset_sec": 0,
"time": 920
}{
"event": "finish",
"reason": "stop",
"time": 5460
}{
"event": "error",
"error": "Concurrency limit exceeded",
"max_concurrency": 8
}API key authentication using Bearer token.
Get your API key from https://fish.audio/app/api-keys
Pass the token in the Authorization header:
Authorization: Bearer YOUR_API_KEY
TTS model to use for this session. If omitted or set to an unrecognized value, the session falls back to s2.1-pro.
s1, s2-pro, s2.1-pro, s2.1-pro-free, drama-3-previewInitiates a TTS streaming session with configuration.
This must be the first message sent after connecting. It contains all the configuration for voice, audio format, and generation parameters.
The request payload uses the same fields as the HTTP TTS API. In
WebSocket mode, request.text is typically empty in the StartEvent and
the actual text is streamed through subsequent TextEvent messages.
For full parameter details, examples, and model-specific guidance, see the HTTP Text to Speech API.
Sends a chunk of text for synthesis.
You can send multiple TextEvent messages in sequence. The server will buffer and synthesize text according to the chunk_length parameter from StartEvent.
Forces immediate synthesis of all buffered text.
Use this when you want audio generated immediately without waiting for more text or for the buffer to fill up. Useful for ensuring low latency in interactive applications.
Signals the end of the text stream.
After sending this event, the server will finish synthesizing any remaining buffered text and send a FinishEvent before closing the connection.
Contains generated audio bytes and the latest word-level alignment snapshot.
Concatenate every audio payload in arrival order to reconstruct the
complete audio. Alignment is cumulative for each chunk_seq, and segment
times are relative to that chunk.
Signals that the timestamped TTS session has completed. After receiving this event, you can send another StartEvent on the same connection.
Reports a request-level failure, such as an invalid StartEvent, a missing voice reference, or a concurrency refusal. The socket closes afterwards.
Was this page helpful?