> ## Documentation Index
> Fetch the complete documentation index at: https://docs.fish.audio/llms.txt
> Use this file to discover all available pages before exploring further.

# Duplex API Converse

> Voice conversation WebSocket with server-side turn detection, streamed speech, and interruption handling.

> [!WARNING]
>
> **Preview — invitation only.** Duplex API access is currently limited to
> selected customers. Public documentation does not grant service access.

Use this endpoint when you want Fish to provide turn detection. Read the [Duplex API guide](https://docs.fish.audio/features/duplex-api) for the architecture, starter application, and conversation flow.

Select a voice with `session.start.voice.reference_id`. See [Voice presets](https://docs.fish.audio/features/duplex-api#choose-a-voice) for Clara and the other ready-to-use reference IDs, or supply a custom Fish Voice Library ID. Use the ID, not the voice's display name.

Every message is a binary WebSocket frame containing a MessagePack map. Audio
payloads use MessagePack binary data, not base64 strings.


## AsyncAPI

````yaml api-reference/duplex-api-asyncapi.yml duplexConverse
id: duplexConverse
title: Duplex converse
description: >
  Full-duplex voice agent channel with server-side turn detection (`turn.eager`,
  `turn.resumed`, `turn.committed`), barge-in interruption, and `heard_text`
  receipts.
servers:
  - id: preview
    protocol: wss
    host: api.fish.audio
    bindings: []
    variables: []
address: /v1/duplex/converse
parameters: []
bindings: []
operations:
  - &ref_9
    id: converseClientFrames
    title: Converse client frames
    description: Client-to-server control and continuous caller audio frames (`C -> S`).
    type: receive
    messages:
      - &ref_11
        id: sessionStart
        contentType: application/msgpack
        payload:
          - name: session.start
            description: >
              Sent **once** as the very first frame within 5 s of connecting (or
              within 1 s when server capacity is full). Configures voice, audio
              format, and eager turn options for the entire call.
            type: object
            properties:
              - name: type
                type: string
                description: session.start
                required: true
              - name: voice
                type: object
                description: >-
                  Optional Fish Voice Library model. Omit `voice` to use the
                  server's default voice.
                required: false
                properties:
                  - name: reference_id
                    type: string
                    description: >-
                      Fish Voice Library model ID (1-128 chars), e.g.
                      `9a9cf47702da476aa4629e2506d4a857`.
                    required: true
              - name: audio
                type: object
                required: true
                properties:
                  - name: encoding
                    type: string
                    description: >-
                      `pcm_s16le` (16-bit signed LE mono) or `mulaw` (8-bit
                      G.711 at 8000 Hz).
                    enumValues:
                      - pcm_s16le
                      - mulaw
                    required: true
                  - name: sample_rate
                    type: integer
                    description: Caller input sample rate in Hz.
                    enumValues:
                      - 8000
                      - 16000
                      - 24000
                      - 44100
                      - 48000
                    required: true
                  - name: output_sample_rate
                    type: integer
                    description: >-
                      Agent output PCM sample rate in Hz (defaults to
                      `sample_rate`).
                    enumValues:
                      - 8000
                      - 16000
                      - 24000
                      - 44100
                      - 48000
                    required: false
                  - name: noise_cancellation
                    type: boolean
                    description: >-
                      Request denoising of caller audio for the service's
                      listening. Availability and listening delay are reported
                      in `session.started.noise_cancellation`. Does not change
                      agent output audio or replace echo cancellation.
                    required: false
              - name: turns
                type: object
                description: >-
                  Conversational turn detection settings (`/v1/duplex/converse`
                  only).
                required: false
                properties:
                  - name: eager
                    type: boolean
                    description: >-
                      **Recommended: `true`.** Emits `turn.eager` and
                      `turn.resumed` for speculative low-latency generation.
                    required: false
                  - name: backchannel
                    type: boolean
                    description: >-
                      Emit `turn.backchannel` when the caller says brief
                      acknowledgements while the agent speaks.
                    required: false
                  - name: interim
                    type: boolean
                    description: >-
                      Emit streaming `turn.interim` ASR drafts for live UI
                      captions.
                    required: false
              - name: own_speech
                type: string
                description: >-
                  `natural` enables conversational prosody/breaths; `quiet`
                  suppresses autonomous sounds.
                enumValues:
                  - natural
                  - quiet
                required: false
              - name: playback
                type: object
                required: false
                properties:
                  - name: delay_ms
                    type: integer
                    description: >-
                      Client speaker/jitter-buffer latency in milliseconds for
                      accurate `heard_text` alignment on barge-in.
                    required: false
              - name: session_ending
                type: boolean
                description: >-
                  Emit `session.ending` warnings at 120 s and 30 s before
                  `max_session_s`.
                required: false
        headers: []
        jsonPayloadSchema:
          type: object
          required:
            - type
            - audio
          properties:
            type:
              type: string
              const: session.start
              x-parser-schema-id: <anonymous-schema-2>
            voice:
              type: object
              required:
                - reference_id
              description: >-
                Optional Fish Voice Library model. Omit `voice` to use the
                server's default voice.
              properties:
                reference_id:
                  type: string
                  description: >-
                    Fish Voice Library model ID (1-128 chars), e.g.
                    `9a9cf47702da476aa4629e2506d4a857`.
                  x-parser-schema-id: <anonymous-schema-4>
              x-parser-schema-id: <anonymous-schema-3>
            audio:
              type: object
              required:
                - encoding
                - sample_rate
              properties:
                encoding:
                  type: string
                  enum:
                    - pcm_s16le
                    - mulaw
                  description: >-
                    `pcm_s16le` (16-bit signed LE mono) or `mulaw` (8-bit G.711
                    at 8000 Hz).
                  x-parser-schema-id: <anonymous-schema-6>
                sample_rate:
                  type: integer
                  enum:
                    - 8000
                    - 16000
                    - 24000
                    - 44100
                    - 48000
                  description: Caller input sample rate in Hz.
                  x-parser-schema-id: <anonymous-schema-7>
                output_sample_rate:
                  type: integer
                  enum:
                    - 8000
                    - 16000
                    - 24000
                    - 44100
                    - 48000
                  description: >-
                    Agent output PCM sample rate in Hz (defaults to
                    `sample_rate`).
                  x-parser-schema-id: <anonymous-schema-8>
                noise_cancellation:
                  type: boolean
                  default: false
                  description: >-
                    Request denoising of caller audio for the service's
                    listening. Availability and listening delay are reported in
                    `session.started.noise_cancellation`. Does not change agent
                    output audio or replace echo cancellation.
                  x-parser-schema-id: <anonymous-schema-9>
              x-parser-schema-id: <anonymous-schema-5>
            turns:
              type: object
              description: >-
                Conversational turn detection settings (`/v1/duplex/converse`
                only).
              properties:
                eager:
                  type: boolean
                  default: false
                  description: >-
                    **Recommended: `true`.** Emits `turn.eager` and
                    `turn.resumed` for speculative low-latency generation.
                  x-parser-schema-id: <anonymous-schema-11>
                backchannel:
                  type: boolean
                  default: false
                  description: >-
                    Emit `turn.backchannel` when the caller says brief
                    acknowledgements while the agent speaks.
                  x-parser-schema-id: <anonymous-schema-12>
                interim:
                  type: boolean
                  default: false
                  description: >-
                    Emit streaming `turn.interim` ASR drafts for live UI
                    captions.
                  x-parser-schema-id: <anonymous-schema-13>
              x-parser-schema-id: <anonymous-schema-10>
            own_speech:
              type: string
              enum:
                - natural
                - quiet
              default: natural
              description: >-
                `natural` enables conversational prosody/breaths; `quiet`
                suppresses autonomous sounds.
              x-parser-schema-id: <anonymous-schema-14>
            playback:
              type: object
              properties:
                delay_ms:
                  type: integer
                  minimum: 0
                  maximum: 2000
                  default: 0
                  description: >-
                    Client speaker/jitter-buffer latency in milliseconds for
                    accurate `heard_text` alignment on barge-in.
                  x-parser-schema-id: <anonymous-schema-16>
              x-parser-schema-id: <anonymous-schema-15>
            session_ending:
              type: boolean
              default: false
              description: >-
                Emit `session.ending` warnings at 120 s and 30 s before
                `max_session_s`.
              x-parser-schema-id: <anonymous-schema-17>
          x-parser-schema-id: <anonymous-schema-1>
        title: session.start
        description: >
          Sent **once** as the very first frame within 5 s of connecting (or
          within 1 s when server capacity is full). Configures voice, audio
          format, and eager turn options for the entire call.
        example: |-
          {
            "type": "session.start",
            "voice": {
              "reference_id": "1c8ce946e6c847e6a3b32a88cce2a1e4"
            },
            "audio": {
              "encoding": "pcm_s16le",
              "sample_rate": 24000,
              "noise_cancellation": true
            },
            "turns": {
              "eager": true,
              "backchannel": true,
              "interim": true
            },
            "playback": {
              "delay_ms": 260
            },
            "session_ending": true
          }
        bindings: []
        extensions:
          - id: x-parser-unique-object-id
            value: sessionStart
      - &ref_12
        id: clientAudio
        contentType: application/msgpack
        payload:
          - name: audio
            description: >
              Continuous real-time caller audio stream (`20 ms` to `100 ms` per
              frame), **silence included**. Acts as the session clock; stopping
              for over 30 s closes the call (`4408 idle_timeout`) unless
              `listen.pause` is active.
            type: object
            properties:
              - name: type
                type: string
                description: audio
                required: true
              - name: data
                type: string
                description: >-
                  Raw MessagePack `bin` audio bytes matching
                  `session.start.audio`.
                required: true
        headers: []
        jsonPayloadSchema:
          type: object
          required:
            - type
            - data
          properties:
            type:
              type: string
              const: audio
              x-parser-schema-id: <anonymous-schema-19>
            data:
              type: string
              format: binary
              description: >-
                Raw MessagePack `bin` audio bytes matching
                `session.start.audio`.
              x-parser-schema-id: <anonymous-schema-20>
          x-parser-schema-id: <anonymous-schema-18>
        title: audio
        description: >
          Continuous real-time caller audio stream (`20 ms` to `100 ms` per
          frame), **silence included**. Acts as the session clock; stopping for
          over 30 s closes the call (`4408 idle_timeout`) unless `listen.pause`
          is active.
        example: |-
          {
            "type": "audio",
            "data": "<960 bytes raw pcm_s16le (20 ms @ 24 kHz)>"
          }
        bindings: []
        extensions:
          - id: x-parser-unique-object-id
            value: clientAudio
      - &ref_13
        id: replyText
        contentType: application/msgpack
        payload:
          - name: reply.text
            description: >
              Streams text chunks from your LLM for the agent to speak. Starting
              a new `reply_id` preempts any currently speaking reply.
            type: object
            properties:
              - name: type
                type: string
                description: reply.text
                required: true
              - name: reply_id
                type: string
                description: Client-assigned reply identifier (1-64 chars).
                required: true
              - name: text
                type: string
                description: >-
                  Spoken text delta (up to 4,096 bytes). May be `""` when
                  signaling `last: true` or `flush: true`.
                required: true
              - name: for
                type: string
                description: >-
                  **First frame of a `reply_id` only.** Names the `eager_id` (or
                  `turn_id`) being answered. When set to an `eager_id`, Fish
                  synthesizes and holds audio silently until `turn.committed`
                  approves it.
                required: false
              - name: flush
                type: boolean
                description: >-
                  Synthesize accumulated text immediately without closing the
                  reply (ideal for short filler lines before running a tool).
                required: false
              - name: last
                type: boolean
                description: Marks the end of this `reply_id` text stream.
                required: false
        headers: []
        jsonPayloadSchema:
          type: object
          required:
            - type
            - reply_id
            - text
          properties:
            type:
              type: string
              const: reply.text
              x-parser-schema-id: <anonymous-schema-22>
            reply_id:
              type: string
              description: Client-assigned reply identifier (1-64 chars).
              x-parser-schema-id: <anonymous-schema-23>
            text:
              type: string
              maxLength: 4096
              description: >-
                Spoken text delta (up to 4,096 bytes). May be `""` when
                signaling `last: true` or `flush: true`.
              x-parser-schema-id: <anonymous-schema-24>
            for:
              type: string
              description: >-
                **First frame of a `reply_id` only.** Names the `eager_id` (or
                `turn_id`) being answered. When set to an `eager_id`, Fish
                synthesizes and holds audio silently until `turn.committed`
                approves it.
              x-parser-schema-id: <anonymous-schema-25>
            flush:
              type: boolean
              default: false
              description: >-
                Synthesize accumulated text immediately without closing the
                reply (ideal for short filler lines before running a tool).
              x-parser-schema-id: <anonymous-schema-26>
            last:
              type: boolean
              default: false
              description: Marks the end of this `reply_id` text stream.
              x-parser-schema-id: <anonymous-schema-27>
          x-parser-schema-id: <anonymous-schema-21>
        title: reply.text
        description: >
          Streams text chunks from your LLM for the agent to speak. Starting a
          new `reply_id` preempts any currently speaking reply.
        example: |-
          {
            "type": "reply.text",
            "reply_id": "r-1-9f3a",
            "for": "e_01",
            "text": "Got it, adding a burger meal. ",
            "flush": true,
            "last": false
          }
        bindings: []
        extensions:
          - id: x-parser-unique-object-id
            value: replyText
      - &ref_14
        id: replyCancel
        contentType: application/msgpack
        payload:
          - name: reply.cancel
            description: >-
              Cancels an in-progress or held speculative reply (for example,
              when `turn.resumed` arrives).
            type: object
            properties:
              - name: type
                type: string
                description: reply.cancel
                required: true
              - name: reply_id
                type: string
                required: true
        headers: []
        jsonPayloadSchema:
          type: object
          required:
            - type
            - reply_id
          properties:
            type:
              type: string
              const: reply.cancel
              x-parser-schema-id: <anonymous-schema-29>
            reply_id:
              type: string
              x-parser-schema-id: <anonymous-schema-30>
          x-parser-schema-id: <anonymous-schema-28>
        title: reply.cancel
        description: >-
          Cancels an in-progress or held speculative reply (for example, when
          `turn.resumed` arrives).
        example: |-
          {
            "type": "reply.cancel",
            "reply_id": "r-1-9f3a"
          }
        bindings: []
        extensions:
          - id: x-parser-unique-object-id
            value: replyCancel
      - &ref_15
        id: listenPause
        contentType: application/msgpack
        payload:
          - name: listen.pause
            description: >-
              Mutes caller input processing (during IVR keypad entry, call
              transfer, or hold) and extends idle timeout from 30 s to 300 s.
            type: object
            properties:
              - name: type
                type: string
                description: listen.pause
                required: true
        headers: []
        jsonPayloadSchema:
          type: object
          required:
            - type
          properties:
            type:
              type: string
              const: listen.pause
              x-parser-schema-id: <anonymous-schema-32>
          x-parser-schema-id: <anonymous-schema-31>
        title: listen.pause
        description: >-
          Mutes caller input processing (during IVR keypad entry, call transfer,
          or hold) and extends idle timeout from 30 s to 300 s.
        example: |-
          {
            "type": "listen.pause"
          }
        bindings: []
        extensions:
          - id: x-parser-unique-object-id
            value: listenPause
      - &ref_16
        id: listenResume
        contentType: application/msgpack
        payload:
          - name: listen.resume
            description: Resumes caller audio listening and restores the 30 s idle timeout.
            type: object
            properties:
              - name: type
                type: string
                description: listen.resume
                required: true
        headers: []
        jsonPayloadSchema:
          type: object
          required:
            - type
          properties:
            type:
              type: string
              const: listen.resume
              x-parser-schema-id: <anonymous-schema-34>
          x-parser-schema-id: <anonymous-schema-33>
        title: listen.resume
        description: Resumes caller audio listening and restores the 30 s idle timeout.
        example: |-
          {
            "type": "listen.resume"
          }
        bindings: []
        extensions:
          - id: x-parser-unique-object-id
            value: listenResume
    bindings: []
    extensions: &ref_8
      - id: x-parser-unique-object-id
        value: duplexConverse
  - &ref_10
    id: converseServerFrames
    title: Converse server frames
    description: >-
      Server-to-client turn, reply receipt, and continuous agent audio frames
      (`S -> C`).
    type: send
    messages:
      - &ref_17
        id: sessionStarted
        contentType: application/msgpack
        payload:
          - name: session.started
            description: >-
              Confirms the session is live, returning active limits, audio
              rates, and LLM prompt `text_rules`. Begin streaming real-time
              caller audio upon receipt.
            type: object
            properties:
              - name: seq
                type: integer
                required: true
              - name: type
                type: string
                description: session.started
                required: true
              - name: session_id
                type: string
                required: true
              - name: limits
                type: object
                required: true
                properties:
                  - name: max_session_s
                    type: integer
                    required: false
                  - name: idle_s
                    type: integer
                    required: false
                  - name: max_text_bytes
                    type: integer
                    required: false
                  - name: max_held_bytes
                    type: integer
                    required: false
                  - name: max_held_s
                    type: integer
                    required: false
                  - name: max_message_bytes
                    type: integer
                    required: false
              - name: text_rules
                type: object
                description: Spoken formatting rules to inject into your LLM system prompt.
                required: true
                properties:
                  - name: numbers
                    type: string
                    enumValues:
                      - spoken
                      - written
                    required: false
                  - name: ack_in_t
                    type: boolean
                    required: false
                  - name: avoid
                    type: array
                    required: false
                    properties:
                      - name: item
                        type: string
                        required: false
              - name: audio
                type: object
                required: true
                properties:
                  - name: encoding
                    type: string
                    required: false
                  - name: sample_rate
                    type: integer
                    required: false
                  - name: output_sample_rate
                    type: integer
                    required: false
              - name: noise_cancellation
                type: object
                description: >-
                  Effective server-side caller denoising status. Off when not
                  requested or unavailable; the session continues either way.
                required: false
                properties:
                  - name: 'on'
                    type: boolean
                    description: Whether caller denoising is active for this session.
                    required: false
                  - name: delay_ms
                    type: integer
                    description: >-
                      Additional listening delay in milliseconds; not an output
                      audio timestamp.
                    required: false
        headers: []
        jsonPayloadSchema:
          type: object
          required:
            - seq
            - type
            - session_id
            - limits
            - text_rules
            - audio
          properties:
            seq:
              type: integer
              x-parser-schema-id: <anonymous-schema-36>
            type:
              type: string
              const: session.started
              x-parser-schema-id: <anonymous-schema-37>
            session_id:
              type: string
              x-parser-schema-id: <anonymous-schema-38>
            limits:
              type: object
              properties:
                max_session_s:
                  type: integer
                  x-parser-schema-id: <anonymous-schema-40>
                idle_s:
                  type: integer
                  x-parser-schema-id: <anonymous-schema-41>
                max_text_bytes:
                  type: integer
                  x-parser-schema-id: <anonymous-schema-42>
                max_held_bytes:
                  type: integer
                  x-parser-schema-id: <anonymous-schema-43>
                max_held_s:
                  type: integer
                  x-parser-schema-id: <anonymous-schema-44>
                max_message_bytes:
                  type: integer
                  x-parser-schema-id: <anonymous-schema-45>
              x-parser-schema-id: <anonymous-schema-39>
            text_rules:
              type: object
              description: Spoken formatting rules to inject into your LLM system prompt.
              properties:
                numbers:
                  type: string
                  enum:
                    - spoken
                    - written
                  x-parser-schema-id: <anonymous-schema-47>
                ack_in_t:
                  type: boolean
                  x-parser-schema-id: <anonymous-schema-48>
                avoid:
                  type: array
                  items:
                    type: string
                    x-parser-schema-id: <anonymous-schema-50>
                  x-parser-schema-id: <anonymous-schema-49>
              x-parser-schema-id: <anonymous-schema-46>
            audio:
              type: object
              properties:
                encoding:
                  type: string
                  x-parser-schema-id: <anonymous-schema-52>
                sample_rate:
                  type: integer
                  x-parser-schema-id: <anonymous-schema-53>
                output_sample_rate:
                  type: integer
                  x-parser-schema-id: <anonymous-schema-54>
              x-parser-schema-id: <anonymous-schema-51>
            noise_cancellation:
              type: object
              description: >-
                Effective server-side caller denoising status. Off when not
                requested or unavailable; the session continues either way.
              properties:
                'on':
                  type: boolean
                  description: Whether caller denoising is active for this session.
                  x-parser-schema-id: <anonymous-schema-56>
                delay_ms:
                  type: integer
                  minimum: 0
                  description: >-
                    Additional listening delay in milliseconds; not an output
                    audio timestamp.
                  x-parser-schema-id: <anonymous-schema-57>
              x-parser-schema-id: <anonymous-schema-55>
          x-parser-schema-id: <anonymous-schema-35>
        title: session.started
        description: >-
          Confirms the session is live, returning active limits, audio rates,
          and LLM prompt `text_rules`. Begin streaming real-time caller audio
          upon receipt.
        example: |-
          {
            "seq": 0,
            "type": "session.started",
            "session_id": "20261116-090715-6462dd",
            "limits": {
              "max_session_s": 1800,
              "idle_s": 30,
              "max_text_bytes": 4096,
              "max_held_bytes": 8192,
              "max_held_s": 15,
              "max_message_bytes": 131072
            },
            "text_rules": {
              "numbers": "written",
              "ack_in_t": false,
              "avoid": [
                "colon",
                "semicolon",
                "spaced_dash"
              ]
            },
            "audio": {
              "encoding": "pcm_s16le",
              "sample_rate": 24000,
              "output_sample_rate": 24000
            },
            "noise_cancellation": {
              "on": true,
              "delay_ms": 28
            }
          }
        bindings: []
        extensions:
          - id: x-parser-unique-object-id
            value: sessionStarted
      - &ref_18
        id: serverAudio
        contentType: application/msgpack
        payload:
          - name: audio
            description: >-
              Continuous `pcm_s16le` mono agent audio stream at
              `output_sample_rate` (silence included). Buffer ~240–300 ms on
              playback.
            type: object
            properties:
              - name: seq
                type: integer
                required: true
              - name: type
                type: string
                description: audio
                required: true
              - name: data
                type: string
                required: true
        headers: []
        jsonPayloadSchema:
          type: object
          required:
            - seq
            - type
            - data
          properties:
            seq:
              type: integer
              x-parser-schema-id: <anonymous-schema-59>
            type:
              type: string
              const: audio
              x-parser-schema-id: <anonymous-schema-60>
            data:
              type: string
              format: binary
              x-parser-schema-id: <anonymous-schema-61>
          x-parser-schema-id: <anonymous-schema-58>
        title: audio
        description: >-
          Continuous `pcm_s16le` mono agent audio stream at `output_sample_rate`
          (silence included). Buffer ~240–300 ms on playback.
        example: |-
          {
            "seq": 1,
            "type": "audio",
            "data": "<raw pcm_s16le mono bytes>"
          }
        bindings: []
        extensions:
          - id: x-parser-unique-object-id
            value: serverAudio
      - &ref_19
        id: turnEager
        contentType: application/msgpack
        payload:
          - name: turn.eager
            description: >
              Speculative end-of-turn signal emitted as soon as the caller
              pauses. Start your LLM stream immediately and send `reply.text`
              with `"for": eager_id` (gate mutating database/order writes until
              `turn.committed`).
            type: object
            properties:
              - name: seq
                type: integer
                required: true
              - name: type
                type: string
                description: turn.eager
                required: true
              - name: eager_id
                type: string
                required: true
              - name: turn_id
                type: string
                required: true
              - name: text
                type: string
                required: true
              - name: agent
                type: object
                required: true
                properties:
                  - name: reply_id
                    type: &ref_0
                      - string
                      - 'null'
                    required: false
                  - name: state
                    type: string
                    enumValues:
                      - idle
                      - pending
                      - speaking
                      - interrupted
                      - completed
                      - partial
                      - failed
                    required: true
                  - name: heard_text
                    type: string
                    required: true
                  - name: basis
                    type: string
                    enumValues:
                      - aligned
                      - estimated
                    required: true
        headers: []
        jsonPayloadSchema:
          type: object
          required:
            - seq
            - type
            - eager_id
            - turn_id
            - text
            - agent
          properties:
            seq:
              type: integer
              x-parser-schema-id: <anonymous-schema-63>
            type:
              type: string
              const: turn.eager
              x-parser-schema-id: <anonymous-schema-64>
            eager_id:
              type: string
              x-parser-schema-id: <anonymous-schema-65>
            turn_id:
              type: string
              x-parser-schema-id: <anonymous-schema-66>
            text:
              type: string
              x-parser-schema-id: <anonymous-schema-67>
            agent: &ref_3
              type: object
              required:
                - state
                - heard_text
                - basis
              properties:
                reply_id:
                  type: *ref_0
                  x-parser-schema-id: <anonymous-schema-68>
                state:
                  type: string
                  enum:
                    - idle
                    - pending
                    - speaking
                    - interrupted
                    - completed
                    - partial
                    - failed
                  x-parser-schema-id: <anonymous-schema-69>
                heard_text:
                  type: string
                  x-parser-schema-id: <anonymous-schema-70>
                basis:
                  type: string
                  enum:
                    - aligned
                    - estimated
                  x-parser-schema-id: <anonymous-schema-71>
              x-parser-schema-id: AgentSnapshot
          x-parser-schema-id: <anonymous-schema-62>
        title: turn.eager
        description: >
          Speculative end-of-turn signal emitted as soon as the caller pauses.
          Start your LLM stream immediately and send `reply.text` with `"for":
          eager_id` (gate mutating database/order writes until
          `turn.committed`).
        example: |-
          {
            "seq": 42,
            "type": "turn.eager",
            "eager_id": "e_01",
            "turn_id": "t_01",
            "text": "Can I get a burger meal",
            "agent": {
              "reply_id": null,
              "state": "idle",
              "heard_text": "",
              "basis": "aligned"
            }
          }
        bindings: []
        extensions:
          - id: x-parser-unique-object-id
            value: turnEager
      - &ref_20
        id: turnResumed
        contentType: application/msgpack
        payload:
          - name: turn.resumed
            description: >-
              Caller continued speaking after `turn.eager`. Cancel your
              in-flight speculative LLM stream and send `reply.cancel`.
            type: object
            properties:
              - name: seq
                type: integer
                required: true
              - name: type
                type: string
                description: turn.resumed
                required: true
              - name: eager_id
                type: string
                required: true
        headers: []
        jsonPayloadSchema:
          type: object
          required:
            - seq
            - type
            - eager_id
          properties:
            seq:
              type: integer
              x-parser-schema-id: <anonymous-schema-73>
            type:
              type: string
              const: turn.resumed
              x-parser-schema-id: <anonymous-schema-74>
            eager_id:
              type: string
              x-parser-schema-id: <anonymous-schema-75>
          x-parser-schema-id: <anonymous-schema-72>
        title: turn.resumed
        description: >-
          Caller continued speaking after `turn.eager`. Cancel your in-flight
          speculative LLM stream and send `reply.cancel`.
        example: |-
          {
            "seq": 45,
            "type": "turn.resumed",
            "eager_id": "e_01"
          }
        bindings: []
        extensions:
          - id: x-parser-unique-object-id
            value: turnResumed
      - &ref_21
        id: turnCommitted
        contentType: application/msgpack
        payload:
          - name: turn.committed
            description: >
              Authoritative finalized caller turn. If `approved_eager_id`
              matches your active speculative run, Fish Audio immediately
              releases the pre-synthesized audio buffer (`lead_ms` reports
              latency saved) and your server can commit any gated write tools.
            type: object
            properties:
              - name: seq
                type: integer
                required: true
              - name: type
                type: string
                description: turn.committed
                required: true
              - name: turn_id
                type: string
                required: true
              - name: text
                type: string
                required: true
              - name: kind
                type: string
                description: >-
                  `turn` is addressed to the agent; `aside` is speech directed
                  away from the phone/mic.
                enumValues:
                  - turn
                  - aside
                required: true
              - name: approved_eager_id
                type: &ref_1
                  - string
                  - 'null'
                required: false
              - name: lead_ms
                type: &ref_2
                  - integer
                  - 'null'
                description: Milliseconds of speculative head-start saved by `turn.eager`.
                required: false
              - name: agent
                type: object
                required: true
                properties:
                  - name: reply_id
                    type: *ref_0
                    required: false
                  - name: state
                    type: string
                    enumValues:
                      - idle
                      - pending
                      - speaking
                      - interrupted
                      - completed
                      - partial
                      - failed
                    required: true
                  - name: heard_text
                    type: string
                    required: true
                  - name: basis
                    type: string
                    enumValues:
                      - aligned
                      - estimated
                    required: true
        headers: []
        jsonPayloadSchema:
          type: object
          required:
            - seq
            - type
            - turn_id
            - text
            - kind
            - agent
          properties:
            seq:
              type: integer
              x-parser-schema-id: <anonymous-schema-77>
            type:
              type: string
              const: turn.committed
              x-parser-schema-id: <anonymous-schema-78>
            turn_id:
              type: string
              x-parser-schema-id: <anonymous-schema-79>
            text:
              type: string
              x-parser-schema-id: <anonymous-schema-80>
            kind:
              type: string
              enum:
                - turn
                - aside
              description: >-
                `turn` is addressed to the agent; `aside` is speech directed
                away from the phone/mic.
              x-parser-schema-id: <anonymous-schema-81>
            approved_eager_id:
              type: *ref_1
              x-parser-schema-id: <anonymous-schema-82>
            lead_ms:
              type: *ref_2
              description: Milliseconds of speculative head-start saved by `turn.eager`.
              x-parser-schema-id: <anonymous-schema-83>
            agent: *ref_3
          x-parser-schema-id: <anonymous-schema-76>
        title: turn.committed
        description: >
          Authoritative finalized caller turn. If `approved_eager_id` matches
          your active speculative run, Fish Audio immediately releases the
          pre-synthesized audio buffer (`lead_ms` reports latency saved) and
          your server can commit any gated write tools.
        example: |-
          {
            "seq": 49,
            "type": "turn.committed",
            "turn_id": "t_01",
            "text": "Can I get a burger meal",
            "kind": "turn",
            "approved_eager_id": "e_01",
            "lead_ms": 280,
            "agent": {
              "reply_id": "r-1-9f3a",
              "state": "pending",
              "heard_text": "",
              "basis": "aligned"
            }
          }
        bindings: []
        extensions:
          - id: x-parser-unique-object-id
            value: turnCommitted
      - &ref_22
        id: turnDropped
        contentType: application/msgpack
        payload:
          - name: turn.dropped
            description: >-
              Sound stopped agent speech or triggered VAD without becoming a
              valid caller turn.
            type: object
            properties:
              - name: seq
                type: integer
                required: true
              - name: type
                type: string
                description: turn.dropped
                required: true
              - name: reason
                type: string
                enumValues:
                  - nonuser
                  - filler
                  - backchannel
                  - echo
                  - no_commit
                  - asr_lost
                required: true
              - name: agent
                type: object
                required: true
                properties:
                  - name: reply_id
                    type: *ref_0
                    required: false
                  - name: state
                    type: string
                    enumValues:
                      - idle
                      - pending
                      - speaking
                      - interrupted
                      - completed
                      - partial
                      - failed
                    required: true
                  - name: heard_text
                    type: string
                    required: true
                  - name: basis
                    type: string
                    enumValues:
                      - aligned
                      - estimated
                    required: true
        headers: []
        jsonPayloadSchema:
          type: object
          required:
            - seq
            - type
            - reason
            - agent
          properties:
            seq:
              type: integer
              x-parser-schema-id: <anonymous-schema-85>
            type:
              type: string
              const: turn.dropped
              x-parser-schema-id: <anonymous-schema-86>
            reason:
              type: string
              enum:
                - nonuser
                - filler
                - backchannel
                - echo
                - no_commit
                - asr_lost
              x-parser-schema-id: <anonymous-schema-87>
            agent: *ref_3
          x-parser-schema-id: <anonymous-schema-84>
        title: turn.dropped
        description: >-
          Sound stopped agent speech or triggered VAD without becoming a valid
          caller turn.
        example: |-
          {
            "seq": 123,
            "type": "<string>",
            "reason": "<string>",
            "agent": {
              "state": "<string>",
              "heard_text": "<string>",
              "basis": "<string>"
            }
          }
        bindings: []
        extensions:
          - id: x-parser-unique-object-id
            value: turnDropped
      - &ref_23
        id: turnBackchannel
        contentType: application/msgpack
        payload:
          - name: turn.backchannel
            description: >-
              Short caller acknowledgement (*"mm-hmm"*, *"okay"*) while the
              agent holds the floor.
            type: object
            properties:
              - name: seq
                type: integer
                required: true
              - name: type
                type: string
                description: turn.backchannel
                required: true
              - name: text
                type: string
                required: true
              - name: at_ms
                type: integer
                required: true
        headers: []
        jsonPayloadSchema:
          type: object
          required:
            - seq
            - type
            - text
            - at_ms
          properties:
            seq:
              type: integer
              x-parser-schema-id: <anonymous-schema-89>
            type:
              type: string
              const: turn.backchannel
              x-parser-schema-id: <anonymous-schema-90>
            text:
              type: string
              x-parser-schema-id: <anonymous-schema-91>
            at_ms:
              type: integer
              x-parser-schema-id: <anonymous-schema-92>
          x-parser-schema-id: <anonymous-schema-88>
        title: turn.backchannel
        description: >-
          Short caller acknowledgement (*"mm-hmm"*, *"okay"*) while the agent
          holds the floor.
        example: |-
          {
            "seq": 123,
            "type": "<string>",
            "text": "<string>",
            "at_ms": 123
          }
        bindings: []
        extensions:
          - id: x-parser-unique-object-id
            value: turnBackchannel
      - &ref_24
        id: turnInterim
        contentType: application/msgpack
        payload:
          - name: turn.interim
            description: >-
              Live streaming ASR partial transcript for UI captions
              (`turns.interim: true`).
            type: object
            properties:
              - name: seq
                type: integer
                required: true
              - name: type
                type: string
                description: turn.interim
                required: true
              - name: turn_id
                type: string
                required: true
              - name: text
                type: string
                required: true
        headers: []
        jsonPayloadSchema:
          type: object
          required:
            - seq
            - type
            - turn_id
            - text
          properties:
            seq:
              type: integer
              x-parser-schema-id: <anonymous-schema-94>
            type:
              type: string
              const: turn.interim
              x-parser-schema-id: <anonymous-schema-95>
            turn_id:
              type: string
              x-parser-schema-id: <anonymous-schema-96>
            text:
              type: string
              x-parser-schema-id: <anonymous-schema-97>
          x-parser-schema-id: <anonymous-schema-93>
        title: turn.interim
        description: >-
          Live streaming ASR partial transcript for UI captions (`turns.interim:
          true`).
        example: |-
          {
            "seq": 123,
            "type": "<string>",
            "turn_id": "<string>",
            "text": "<string>"
          }
        bindings: []
        extensions:
          - id: x-parser-unique-object-id
            value: turnInterim
      - &ref_25
        id: replyStarted
        contentType: application/msgpack
        payload:
          - name: reply.started
            description: >-
              Emitted when the first audible speech sample of `reply_id` enters
              the output audio stream.
            type: object
            properties:
              - name: seq
                type: integer
                required: true
              - name: type
                type: string
                description: reply.started
                required: true
              - name: reply_id
                type: string
                required: true
              - name: at_ms
                type: integer
                description: >-
                  Session output timestamp (`sample_index * 1000 /
                  output_sample_rate`).
                required: true
        headers: []
        jsonPayloadSchema:
          type: object
          required:
            - seq
            - type
            - reply_id
            - at_ms
          properties:
            seq:
              type: integer
              x-parser-schema-id: <anonymous-schema-99>
            type:
              type: string
              const: reply.started
              x-parser-schema-id: <anonymous-schema-100>
            reply_id:
              type: string
              x-parser-schema-id: <anonymous-schema-101>
            at_ms:
              type: integer
              description: >-
                Session output timestamp (`sample_index * 1000 /
                output_sample_rate`).
              x-parser-schema-id: <anonymous-schema-102>
          x-parser-schema-id: <anonymous-schema-98>
        title: reply.started
        description: >-
          Emitted when the first audible speech sample of `reply_id` enters the
          output audio stream.
        example: |-
          {
            "seq": 123,
            "type": "<string>",
            "reply_id": "<string>",
            "at_ms": 123
          }
        bindings: []
        extensions:
          - id: x-parser-unique-object-id
            value: replyStarted
      - &ref_26
        id: replyProgress
        contentType: application/msgpack
        payload:
          - name: reply.progress
            description: Periodic spoken character progress along `reply_id`.
            type: object
            properties:
              - name: seq
                type: integer
                required: true
              - name: type
                type: string
                description: reply.progress
                required: true
              - name: reply_id
                type: string
                required: true
              - name: spoken_chars
                type: integer
                required: true
              - name: basis
                type: string
                enumValues:
                  - aligned
                  - estimated
                required: true
        headers: []
        jsonPayloadSchema:
          type: object
          required:
            - seq
            - type
            - reply_id
            - spoken_chars
            - basis
          properties:
            seq:
              type: integer
              x-parser-schema-id: <anonymous-schema-104>
            type:
              type: string
              const: reply.progress
              x-parser-schema-id: <anonymous-schema-105>
            reply_id:
              type: string
              x-parser-schema-id: <anonymous-schema-106>
            spoken_chars:
              type: integer
              x-parser-schema-id: <anonymous-schema-107>
            basis:
              type: string
              enum:
                - aligned
                - estimated
              x-parser-schema-id: <anonymous-schema-108>
          x-parser-schema-id: <anonymous-schema-103>
        title: reply.progress
        description: Periodic spoken character progress along `reply_id`.
        example: |-
          {
            "seq": 123,
            "type": "<string>",
            "reply_id": "<string>",
            "spoken_chars": 123,
            "basis": "<string>"
          }
        bindings: []
        extensions:
          - id: x-parser-unique-object-id
            value: replyProgress
      - &ref_27
        id: replyInterrupted
        contentType: application/msgpack
        payload:
          - name: reply.interrupted
            description: >-
              Emitted up to twice when a reply is cut short (`final: false` at
              cut onset for immediate client speaker fade-out; `final: true`
              once agent audio settles).
            type: object
            properties:
              - name: seq
                type: integer
                required: true
              - name: type
                type: string
                description: reply.interrupted
                required: true
              - name: reply_id
                type: string
                required: true
              - name: by
                type: string
                enumValues:
                  - caller
                  - client
                required: true
              - name: heard_text
                type: string
                required: true
              - name: basis
                type: string
                enumValues:
                  - aligned
                  - estimated
                required: true
              - name: final
                type: boolean
                required: true
        headers: []
        jsonPayloadSchema:
          type: object
          required:
            - seq
            - type
            - reply_id
            - by
            - heard_text
            - basis
            - final
          properties:
            seq:
              type: integer
              x-parser-schema-id: <anonymous-schema-110>
            type:
              type: string
              const: reply.interrupted
              x-parser-schema-id: <anonymous-schema-111>
            reply_id:
              type: string
              x-parser-schema-id: <anonymous-schema-112>
            by:
              type: string
              enum:
                - caller
                - client
              x-parser-schema-id: <anonymous-schema-113>
            heard_text:
              type: string
              x-parser-schema-id: <anonymous-schema-114>
            basis:
              type: string
              enum:
                - aligned
                - estimated
              x-parser-schema-id: <anonymous-schema-115>
            final:
              type: boolean
              x-parser-schema-id: <anonymous-schema-116>
          x-parser-schema-id: <anonymous-schema-109>
        title: reply.interrupted
        description: >-
          Emitted up to twice when a reply is cut short (`final: false` at cut
          onset for immediate client speaker fade-out; `final: true` once agent
          audio settles).
        example: |-
          {
            "seq": 123,
            "type": "<string>",
            "reply_id": "<string>",
            "by": "<string>",
            "heard_text": "<string>",
            "basis": "<string>",
            "final": true
          }
        bindings: []
        extensions:
          - id: x-parser-unique-object-id
            value: replyInterrupted
      - &ref_28
        id: replyDone
        contentType: application/msgpack
        payload:
          - name: reply.done
            description: >
              Single authoritative receipt concluding every reply. Always append
              `heard_text` (never unplayed `unheard_text`) to your LLM
              conversation history.
            type: object
            properties:
              - name: seq
                type: integer
                required: true
              - name: type
                type: string
                description: reply.done
                required: true
              - name: reply_id
                type: string
                required: true
              - name: status
                type: string
                enumValues:
                  - completed
                  - interrupted
                  - partial
                  - replaced
                  - rejected
                  - failed
                required: true
              - name: reason
                type: &ref_4
                  - string
                  - 'null'
                description: >-
                  `not_approved` | `stale_turn` | `held_timeout` |
                  `held_overflow` | `stopped_early` | `never_spoke` |
                  `empty_text` | `text_refused` | `engine_lost` |
                  `session_ended`
                required: false
              - name: by
                type: &ref_5
                  - string
                  - 'null'
                enumValues:
                  - caller
                  - client
                  - replaced
                required: false
              - name: heard_text
                type: string
                description: Exact prefix heard by the caller.
                required: true
              - name: unheard_text
                type: string
                description: >-
                  Remaining submitted text that never played (`heard_text +
                  unheard_text` equals total submitted text).
                required: true
              - name: heard_chars
                type: integer
                required: true
              - name: basis
                type: string
                enumValues:
                  - aligned
                  - estimated
                required: true
              - name: audio_end_ms
                type: &ref_6
                  - integer
                  - 'null'
                required: false
        headers: []
        jsonPayloadSchema:
          type: object
          required:
            - seq
            - type
            - reply_id
            - status
            - heard_text
            - unheard_text
            - heard_chars
            - basis
          properties:
            seq:
              type: integer
              x-parser-schema-id: <anonymous-schema-118>
            type:
              type: string
              const: reply.done
              x-parser-schema-id: <anonymous-schema-119>
            reply_id:
              type: string
              x-parser-schema-id: <anonymous-schema-120>
            status:
              type: string
              enum:
                - completed
                - interrupted
                - partial
                - replaced
                - rejected
                - failed
              x-parser-schema-id: <anonymous-schema-121>
            reason:
              type: *ref_4
              description: >-
                `not_approved` | `stale_turn` | `held_timeout` | `held_overflow`
                | `stopped_early` | `never_spoke` | `empty_text` |
                `text_refused` | `engine_lost` | `session_ended`
              x-parser-schema-id: <anonymous-schema-122>
            by:
              type: *ref_5
              enum:
                - caller
                - client
                - replaced
                - null
              x-parser-schema-id: <anonymous-schema-123>
            heard_text:
              type: string
              description: Exact prefix heard by the caller.
              x-parser-schema-id: <anonymous-schema-124>
            unheard_text:
              type: string
              description: >-
                Remaining submitted text that never played (`heard_text +
                unheard_text` equals total submitted text).
              x-parser-schema-id: <anonymous-schema-125>
            heard_chars:
              type: integer
              x-parser-schema-id: <anonymous-schema-126>
            basis:
              type: string
              enum:
                - aligned
                - estimated
              x-parser-schema-id: <anonymous-schema-127>
            audio_end_ms:
              type: *ref_6
              x-parser-schema-id: <anonymous-schema-128>
          x-parser-schema-id: <anonymous-schema-117>
        title: reply.done
        description: >
          Single authoritative receipt concluding every reply. Always append
          `heard_text` (never unplayed `unheard_text`) to your LLM conversation
          history.
        example: |-
          {
            "seq": 88,
            "type": "reply.done",
            "reply_id": "r-1-9f3a",
            "status": "interrupted",
            "reason": null,
            "by": "caller",
            "heard_text": "Got it, adding a burger meal.",
            "unheard_text": " What drink would you like with that?",
            "heard_chars": 30,
            "basis": "aligned",
            "audio_end_ms": 4120
          }
        bindings: []
        extensions:
          - id: x-parser-unique-object-id
            value: replyDone
      - &ref_29
        id: agentSpeech
        contentType: application/msgpack
        payload:
          - name: agent.speech
            description: >-
              Autonomous conversational backchannel or natural vocalization
              produced by the voice model outside explicit replies.
            type: object
            properties:
              - name: seq
                type: integer
                required: true
              - name: type
                type: string
                description: agent.speech
                required: true
              - name: kind
                type: string
                enumValues:
                  - backchannel
                  - own
                required: true
              - name: start_ms
                type: integer
                required: true
              - name: end_ms
                type: integer
                required: true
        headers: []
        jsonPayloadSchema:
          type: object
          required:
            - seq
            - type
            - kind
            - start_ms
            - end_ms
          properties:
            seq:
              type: integer
              x-parser-schema-id: <anonymous-schema-130>
            type:
              type: string
              const: agent.speech
              x-parser-schema-id: <anonymous-schema-131>
            kind:
              type: string
              enum:
                - backchannel
                - own
              x-parser-schema-id: <anonymous-schema-132>
            start_ms:
              type: integer
              x-parser-schema-id: <anonymous-schema-133>
            end_ms:
              type: integer
              x-parser-schema-id: <anonymous-schema-134>
          x-parser-schema-id: <anonymous-schema-129>
        title: agent.speech
        description: >-
          Autonomous conversational backchannel or natural vocalization produced
          by the voice model outside explicit replies.
        example: |-
          {
            "seq": 123,
            "type": "<string>",
            "kind": "<string>",
            "start_ms": 123,
            "end_ms": 123
          }
        bindings: []
        extensions:
          - id: x-parser-unique-object-id
            value: agentSpeech
      - &ref_30
        id: sessionEnding
        contentType: application/msgpack
        payload:
          - name: session.ending
            description: Warning emitted 120 s and 30 s before `limits.max_session_s`.
            type: object
            properties:
              - name: seq
                type: integer
                required: true
              - name: type
                type: string
                description: session.ending
                required: true
              - name: in_s
                type: integer
                required: true
        headers: []
        jsonPayloadSchema:
          type: object
          required:
            - seq
            - type
            - in_s
          properties:
            seq:
              type: integer
              x-parser-schema-id: <anonymous-schema-136>
            type:
              type: string
              const: session.ending
              x-parser-schema-id: <anonymous-schema-137>
            in_s:
              type: integer
              x-parser-schema-id: <anonymous-schema-138>
          x-parser-schema-id: <anonymous-schema-135>
        title: session.ending
        description: Warning emitted 120 s and 30 s before `limits.max_session_s`.
        example: |-
          {
            "seq": 123,
            "type": "<string>",
            "in_s": 123
          }
        bindings: []
        extensions:
          - id: x-parser-unique-object-id
            value: sessionEnding
      - &ref_31
        id: errorEvent
        contentType: application/msgpack
        payload:
          - name: error
            description: Recoverable or fatal protocol/subsystem error notification.
            type: object
            properties:
              - name: seq
                type: integer
                required: true
              - name: type
                type: string
                description: error
                required: true
              - name: code
                type: string
                required: true
              - name: message
                type: string
                required: true
              - name: recoverable
                type: boolean
                required: true
              - name: lane
                type: &ref_7
                  - string
                  - 'null'
                enumValues:
                  - caller
                  - agent
                required: false
        headers: []
        jsonPayloadSchema:
          type: object
          required:
            - seq
            - type
            - code
            - message
            - recoverable
          properties:
            seq:
              type: integer
              x-parser-schema-id: <anonymous-schema-140>
            type:
              type: string
              const: error
              x-parser-schema-id: <anonymous-schema-141>
            code:
              type: string
              x-parser-schema-id: <anonymous-schema-142>
            message:
              type: string
              x-parser-schema-id: <anonymous-schema-143>
            recoverable:
              type: boolean
              x-parser-schema-id: <anonymous-schema-144>
            lane:
              type: *ref_7
              enum:
                - caller
                - agent
                - null
              x-parser-schema-id: <anonymous-schema-145>
          x-parser-schema-id: <anonymous-schema-139>
        title: error
        description: Recoverable or fatal protocol/subsystem error notification.
        example: |-
          {
            "seq": 123,
            "type": "<string>",
            "code": "<string>",
            "message": "<string>",
            "recoverable": true
          }
        bindings: []
        extensions:
          - id: x-parser-unique-object-id
            value: errorEvent
    bindings: []
    extensions: *ref_8
sendOperations:
  - *ref_9
receiveOperations:
  - *ref_10
sendMessages:
  - *ref_11
  - *ref_12
  - *ref_13
  - *ref_14
  - *ref_15
  - *ref_16
receiveMessages:
  - *ref_17
  - *ref_18
  - *ref_19
  - *ref_20
  - *ref_21
  - *ref_22
  - *ref_23
  - *ref_24
  - *ref_25
  - *ref_26
  - *ref_27
  - *ref_28
  - *ref_29
  - *ref_30
  - *ref_31
extensions:
  - id: x-parser-unique-object-id
    value: duplexConverse
securitySchemes:
  - id: bearerAuth
    name: bearerAuth
    type: http
    description: >
      Pass your Fish Audio API key on the server-to-server WebSocket upgrade
      request:

      `Authorization: Bearer YOUR_API_KEY`
    scheme: bearer
    extensions: []

````

This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.