# Create Model Source: https://docs.fish.audio/api-reference/endpoint/model/create-model post /model Create a new voice model Since this endpoint uploads files, use `multipart/form-data` for regular REST requests. Let your HTTP client set the multipart `Content-Type` boundary automatically. # Delete Model Source: https://docs.fish.audio/api-reference/endpoint/model/delete-model delete /model/{id} Delete an existing model # Get Model Source: https://docs.fish.audio/api-reference/endpoint/model/get-model get /model/{id} Get details of a specific model # List Models Source: https://docs.fish.audio/api-reference/endpoint/model/list-models get /model Get a list of all models # Update Model Source: https://docs.fish.audio/api-reference/endpoint/model/update-model patch /model/{id} Update an existing model # Speech to Text Source: https://docs.fish.audio/api-reference/endpoint/openapi-v1/speech-to-text post /v1/asr Transcribe audio to text This BETA endpoint only accepts `application/form-data` and `application/msgpack`. # Text to Speech Source: https://docs.fish.audio/api-reference/endpoint/openapi-v1/text-to-speech post /v1/tts Convert text to speech This endpoint only accepts `application/json` and `application/msgpack`. For best results, upload reference audio using the [create model](/api-reference/endpoint/model/create-model) before using this one. This improves speech quality and reduces latency. To upload audio clips directly, without pre-uploading, serialize the request body with MessagePack as per the [instructions](/features/text-to-speech#direct-api-messagepack). Audio formats supported: * WAV / PCM * Sample Rate: 8kHz, 16kHz, 24kHz, 32kHz, 44.1kHz * Default Sample Rate: 44.1kHz * 16-bit, mono * MP3 * Sample Rate: 32kHz, 44.1kHz * Default Sample Rate: 44.1kHz * mono * Bitrate: 64kbps, 128kbps (default), 192kbps * Opus * Sample Rate: 48kHz * Default Sample Rate: 48kHz * mono * Bitrate: -1000 (auto), 24kbps, 32kbps (default), 48kbps, 64kbps # Text to Speech Stream with Timestamps Source: https://docs.fish.audio/api-reference/endpoint/openapi-v1/text-to-speech-stream-with-timestamps post /v1/tts/stream/with-timestamp Stream generated speech with timestamp alignment snapshots This endpoint returns `text/event-stream`. Each SSE `message` event contains one JSON payload with a base64-encoded audio chunk. Use this endpoint when you need both progressive audio delivery and text-to-audio alignment data, such as karaoke-style highlighting, word or phrase progress indicators, captions synchronized to generated speech, or timeline editing. ## How the Stream Works The response is a Server-Sent Events stream. Every event includes: | Field | Type | Description | | ------------------------ | ---------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | | `audio_base64` | `string` | One base64-encoded audio chunk. Concatenate all chunks in arrival order to reconstruct the complete audio. | | `content` | `string` | Text content described by this event's latest alignment snapshot. Long input can be split into multiple content chunks. | | `alignment` | `object \| null` | Latest cumulative timestamp snapshot for `chunk_seq`. When present, replace the previous snapshot for that `chunk_seq`; do not append segments. | | `chunk_seq` | `integer` | Sequence number of the text chunk described by `alignment`. Bucket alignment snapshots by this value. | | `chunk_audio_offset_sec` | `number` | Absolute start time of this text chunk within the full audio, in seconds. Add this to segment-local `start` and `end` values for a global audio timeline. | `audio_base64` is the transport stream. `alignment` is a metadata snapshot for `chunk_seq`. They are delivered together in the same SSE event, but the alignment is not a per-audio-packet delta. When `latency` is set to `balanced`, long input can be split into several text chunks. A chunk may produce multiple non-null alignment snapshots as more audio is rendered. Each newer snapshot supersedes the previous snapshot for the same `chunk_seq`. Store alignments in a map keyed by `chunk_seq`. On every non-null `alignment`, replace the stored value for that key. Do not collect every non-null alignment as a separate final result. ## Alignment Shape Each non-null `alignment` contains the current cumulative timing segments for a single text chunk: ```json theme={null} { "audio_base64": "SUQzBAAAAAAA...", "content": "Hello world", "chunk_seq": 0, "chunk_audio_offset_sec": 0.0, "alignment": { "audio_duration": 0.86, "segments": [ { "text": "Hello", "start": 0, "end": 0.42 }, { "text": "world", "start": 0.42, "end": 0.86 } ] } } ``` `start` and `end` are measured in seconds from the start of that text chunk's generated audio. Add `chunk_audio_offset_sec` to get timestamps on the complete audio timeline. `alignment` can be `null` before the first snapshot is available or when alignment is unavailable. After a snapshot exists, later audio events may repeat the latest snapshot so clients can continue using a simple latest-wins update model. ## Minimal Request ```bash theme={null} curl --no-buffer --request POST \ --url https://api.fish.audio/v1/tts/stream/with-timestamp \ --header 'Authorization: Bearer ' \ --header 'Content-Type: application/json' \ --header 'model: s2-pro' \ --data '{ "text": "Hello! Welcome to Fish Audio.", "reference_id": "model-id", "format": "opus", "latency": "balanced" }' ``` ## Parsing the Stream The stream payload uses standard SSE framing. Parse each `data:` line as JSON, append every decoded `audio_base64` chunk to your audio buffer, and replace the latest alignment snapshot for `chunk_seq` whenever `alignment` is non-null. ```python theme={null} import base64 import json import requests response = requests.post( "https://api.fish.audio/v1/tts/stream/with-timestamp", headers={ "Authorization": "Bearer ", "Content-Type": "application/json", "model": "s2-pro", }, json={ "text": "Hello! Welcome to Fish Audio.", "reference_id": "model-id", "format": "opus", "latency": "balanced", }, stream=True, ) audio_chunks = [] alignment_by_chunk = {} for line in response.iter_lines(decode_unicode=True): if not line or not line.startswith("data: "): continue event = json.loads(line.removeprefix("data: ")) audio_chunks.append(base64.b64decode(event["audio_base64"])) if event["alignment"] is not None: alignment_by_chunk[event["chunk_seq"]] = { "content": event["content"], "offset": event["chunk_audio_offset_sec"], "alignment": event["alignment"], } audio = b"".join(audio_chunks) ``` ```javascript theme={null} const response = await fetch( "https://api.fish.audio/v1/tts/stream/with-timestamp", { method: "POST", headers: { Authorization: "Bearer ", "Content-Type": "application/json", model: "s2-pro", }, body: JSON.stringify({ text: "Hello! Welcome to Fish Audio.", reference_id: "model-id", format: "opus", latency: "balanced", }), } ); const audioChunks = []; const alignmentByChunk = new Map(); const decoder = new TextDecoder(); let buffer = ""; for await (const chunk of response.body) { buffer += decoder.decode(chunk, { stream: true }); const events = buffer.split("\n\n"); buffer = events.pop() ?? ""; for (const eventText of events) { const dataLine = eventText .split("\n") .find((line) => line.startsWith("data: ")); if (!dataLine) continue; const event = JSON.parse(dataLine.slice(6)); audioChunks.push(Buffer.from(event.audio_base64, "base64")); if (event.alignment !== null) { alignmentByChunk.set(event.chunk_seq, { content: event.content, offset: event.chunk_audio_offset_sec, alignment: event.alignment, }); } } } const audio = Buffer.concat(audioChunks); ``` ## Handling Split Content Chunks Long input can produce multiple text chunks. Treat audio and alignment as two related streams: 1. Append every decoded `audio_base64` chunk in event order. Do this even when `alignment` is `null`. 2. For non-null `alignment`, replace the stored snapshot for `chunk_seq`. 3. Convert each snapshot's local segment times into global times by adding `chunk_audio_offset_sec`. `audio_base64` chunks are transport chunks, not sentence or word boundaries. Do not try to align each audio chunk individually. Use `alignment.segments` plus `chunk_audio_offset_sec` for text timing. For example, if an event has `chunk_audio_offset_sec: 16.24`, add `16.24` seconds to every segment in that event's `alignment` before rendering it on the complete audio timeline. ```python theme={null} def build_global_timeline(alignment_by_chunk): timeline = [] for chunk_seq, item in sorted(alignment_by_chunk.items()): offset_seconds = item["offset"] alignment = item["alignment"] for segment in alignment["segments"]: timeline.append({ "text": segment["text"], "start": segment["start"] + offset_seconds, "end": segment["end"] + offset_seconds, "chunk_seq": chunk_seq, }) return timeline ``` ```javascript theme={null} function buildGlobalTimeline(alignmentByChunk) { const timeline = []; for (const [chunkSeq, item] of [...alignmentByChunk.entries()].sort( ([a], [b]) => a - b )) { for (const segment of item.alignment.segments) { timeline.push({ text: segment.text, start: segment.start + item.offset, end: segment.end + item.offset, chunk_seq: chunkSeq, }); } } return timeline; } ``` ## Format Guidance For timestamped streaming, we recommend `opus` with the default 48 kHz sample rate when your client supports it. Opus is designed for streaming and gives the best balance of quality, latency, and bandwidth for this endpoint. `wav` and `pcm` avoid lossy codec artifacts and are straightforward to align, but they produce much larger payloads. Use them when you need uncompressed audio, direct sample-level processing, or a playback pipeline that already expects raw audio. Use `mp3` only when broad playback compatibility is more important than the cleanest streaming boundaries. MP3 encoding uses overlapping audio windows, so its encoded chunks may not line up as neatly with timestamp snapshot updates as Opus. This endpoint accepts the same TTS request fields as the [Text to Speech API](/api-reference/endpoint/openapi-v1/text-to-speech), including `reference_id`, `references`, `prosody`, `temperature`, `top_p`, `chunk_length`, `format`, and `latency`. # Voice Design Source: https://docs.fish.audio/api-reference/endpoint/openapi-v1/voice-design post /v1/voice-design Generate candidate voices from a prompt This endpoint only accepts `application/json`. You must include the `model: voice-design-1` header. Extra request fields are rejected. A successful request returns generated voice candidates with `audio_base64` audio payloads. Decode the base64 value to write the candidate audio to a file. ## Example ```bash theme={null} curl --request POST https://api.fish.audio/v1/voice-design \ --header "Authorization: Bearer $FISH_API_KEY" \ --header "Content-Type: application/json" \ --header "model: voice-design-1" \ --data '{ "instruction": "Warm, confident studio narrator with a natural tone", "reference_text": "Welcome to Fish Audio.", "language": "en", "n": 2, "speed": 1, "num_step": 32, "guidance_scale": 2, "instruct_guidance_scale": 0, "seed": 42 }' ``` ## Usage notes * `instruction` is required and must be 1 to 2000 characters. * `reference_text` is optional preview text and can be up to 150 characters. * `n` controls how many candidates are returned. The supported range is 1 to 4. * `seed` is optional and can help reproduce candidate generation. * The endpoint is stateless: it does not create batches, samples, voice models, or presigned URLs. * Billing happens once per successful generation request, not once per candidate. # Get API Credit Source: https://docs.fish.audio/api-reference/endpoint/wallet/get-api-credit get /wallet/{user_id}/api-credit Get current API credit balance # Get User Package Source: https://docs.fish.audio/api-reference/endpoint/wallet/get-user-package get /wallet/{user_id}/package Get current user premium information # WebSocket TTS Streaming Source: https://docs.fish.audio/api-reference/endpoint/websocket/tts-live Real-time text-to-speech streaming via WebSocket The WebSocket TTS endpoint enables bidirectional streaming for low-latency text-to-speech generation with MessagePack serialization. The `request` payload inside `StartEvent` uses the same parameters as the HTTP [Text to Speech API](/api-reference/endpoint/openapi-v1/text-to-speech). For more detailed field guidance, model-specific behavior, and examples, see that page. In WebSocket mode, `request.text` is typically empty in `StartEvent`, and the text content is sent through subsequent `TextEvent` messages. # Errors Source: https://docs.fish.audio/api-reference/errors HTTP status codes, the error response shape, and how to handle them in any language Every Fish Audio error comes back as JSON with a `message` and a `status`: ```json theme={null} { "message": "Invalid Token", "status": 401 } ``` (A request whose body can't be parsed returns a plain-text parse error instead.) ## Status codes | Status | Meaning | What to do | | ------ | -------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- | | `400` | Bad request — invalid parameters, or a `reference_id` / voice that doesn't exist | Fix the request; read the `message`. | | `401` | Invalid or missing API key | Send `Authorization: Bearer `; check it on [API Keys](https://fish.audio/app/api-keys). | | `402` | Insufficient credits | Top up on [Billing](https://fish.audio/app/billing). | | `403` | Not permitted for this key/resource | Check the key's scope and the resource owner. | | `404` | Model or voice not found | Verify the `model_id` / `reference_id`. | | `429` | Rate limit exceeded | Back off and retry (see below). | | `5xx` | Server error | Retry with backoff; if it persists, contact support. | ## Retries Retry `429` and `5xx` with exponential backoff. Don't retry other `4xx` codes — they won't succeed without a change to the request. ```python theme={null} import time from fishaudio import FishAudio from fishaudio.exceptions import RateLimitError, APIError client = FishAudio() for attempt in range(5): try: audio = client.tts.convert(text="Hello!") break except RateLimitError: time.sleep(2 ** attempt) # 1s, 2s, 4s, ... except APIError as e: if e.status >= 500: time.sleep(2 ** attempt) else: raise # 4xx — fix the request ``` ## Handling errors in the SDKs Both SDKs raise typed exceptions you can branch on. The base class carries the status and the parsed body. ```python Python theme={null} from fishaudio.exceptions import ( AuthenticationError, # 401 RateLimitError, # 429 NotFoundError, # 404 APIError, # any other HTTP error — has .status and .message FishAudioError, # base class for all SDK errors ) try: audio = client.tts.convert(text="Hello!") except AuthenticationError: ... # invalid or missing key except RateLimitError: ... # back off and retry except NotFoundError: ... # bad reference_id / model id except APIError as e: print(e.status, e.message) # 400, 402, 5xx, ... ``` ```javascript JavaScript theme={null} import { UnauthorizedError, // 401 TooEarlyError, // 429 NotFoundError, // 404 BadRequestError, // 400 UnprocessableEntityError, // 422 FishAudioError, // base — has .statusCode and .body } from "fish-audio"; try { await client.textToSpeech.convert({ text: "Hello!" }, "s2-pro"); } catch (err) { if (err instanceof UnauthorizedError) { // invalid or missing key } else if (err instanceof TooEarlyError) { // back off and retry } else if (err instanceof FishAudioError) { console.error(err.statusCode, err.body); } } ``` Audio playback via `play()` needs `ffmpeg`. If it's missing, the Python SDK raises `DependencyError` — install `ffmpeg` or save the audio to a file instead. # Introduction Source: https://docs.fish.audio/api-reference/introduction How to use the Fish Audio API ## Welcome You can generate a new API key at [https://fish.audio/app/api-keys/](https://fish.audio/app/api-keys/). ## Quick Start See our [Quick Start](/developer-guide/getting-started/quickstart) guide to generate audio in under 2 minutes. ## Errors Every error returns a JSON body with a `message` and a `status`. See [Errors](/api-reference/errors) for the full status-code table, retry guidance, and SDK exception handling. ## OpenAPI Schema Fish Audio publishes a canonical OpenAPI schema at [https://api.fish.audio/openapi.json](https://api.fish.audio/openapi.json). When working with AI coding agents or IDE assistants, mention this schema URL as part of your prompt or project context so the agent can understand Fish Audio's endpoints, request and response models, authentication requirements, and supported parameters directly from the machine-readable API contract. ## Distributed Tracing Fish Audio inference APIs accept the W3C `traceparent` header so your business-side trace and Fish Audio's inference-side trace can share the same trace ID. See [Tracing & Performance Analysis](/api-reference/observability) for supported endpoints, examples, and enterprise performance analysis details. ## Create a Voice Clone Use our [/model endpoint](/api-reference/endpoint/model/create-model) to create a voice clone model. ## Generate Speech Use our [/v1/tts endpoint](/api-reference/endpoint/openapi-v1/text-to-speech) to generate speech. ## Design a Voice Use our [/v1/voice-design endpoint](/api-reference/endpoint/openapi-v1/voice-design) to generate candidate voices from a prompt. ## Real-time Streaming Use our [Python SDK](/features/realtime-streaming) or [JavaScript SDK](/features/realtime-streaming) for real-time audio streaming with WebSocket. ## Rate Limits You can find the rate limits for each endpoint in the [Rate Limits](/developer-guide/models-pricing/pricing-and-rate-limits) section. # Tracing & Performance Analysis Source: https://docs.fish.audio/api-reference/observability Correlate Fish Audio inference spans with your own distributed traces Fish Audio inference APIs accept the standard W3C Trace Context `traceparent` header. Send this header when your application already has an active trace and you want Fish Audio edge, inference, alignment, and upstream ASR spans to appear under the same trace ID. Supported inference surfaces: * `POST /v1/tts` * `POST /v1/tts/stream/with-timestamp` * `POST /v1/asr` * `wss://api.fish.audio/v1/tts/live` If `traceparent` is omitted or invalid, Fish Audio starts a new trace for the request. Tracing does not change authentication, rate limits, billing, request priority, or generated output. ## Header Format Use the W3C `traceparent` format: ```text theme={null} traceparent: 00--- ``` Example: ```text theme={null} traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01 ``` In this example, `4bf92f3577b34da6a3ce929d0e0e4736` is the trace ID. Fish Audio continues that trace ID when it calls the inference backend and related services. Prefer letting your tracing SDK inject `traceparent` instead of hand-building it. For each Fish Audio request, create a child span in your business service and inject that span context into the outgoing HTTP or WebSocket headers. ## Bind Business and Inference Traces To bind your business-side trace to Fish Audio's inference-side trace: 1. Start or continue a trace in your application for the user workflow. 2. Create a child span around the Fish Audio request. 3. Inject that span context into the request headers. 4. Send the request with `traceparent`. 5. Use the same trace ID in your observability tool to inspect both your business spans and Fish Audio inference spans. For multiple Fish Audio calls in one workflow, keep the same trace ID by using the same parent trace, but let your tracing SDK create a fresh parent/span ID for each outgoing request. ## REST Example ```bash theme={null} TRACEPARENT="00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01" curl --request POST https://api.fish.audio/v1/tts \ --header "Authorization: Bearer $FISH_API_KEY" \ --header "Content-Type: application/json" \ --header "model: s2-pro" \ --header "traceparent: $TRACEPARENT" \ --data '{ "text": "Hello from a traced Fish Audio request.", "reference_id": "model-id", "format": "mp3" }' \ --output out.mp3 ``` ## OpenTelemetry Examples ```python theme={null} import os import requests from opentelemetry import trace from opentelemetry.propagate import inject tracer = trace.get_tracer("my-service") with tracer.start_as_current_span("fish_audio.tts"): headers = { "Authorization": f"Bearer {os.environ['FISH_API_KEY']}", "Content-Type": "application/json", "model": "s2-pro", } inject(headers) response = requests.post( "https://api.fish.audio/v1/tts", headers=headers, json={ "text": "Hello from a traced request.", "reference_id": "model-id", "format": "mp3", }, timeout=60, ) response.raise_for_status() ``` ```javascript theme={null} import { context, propagation, trace } from "@opentelemetry/api"; const tracer = trace.getTracer("my-service"); const span = tracer.startSpan("fish_audio.tts"); const ctx = trace.setSpan(context.active(), span); try { await context.with(ctx, async () => { const headers = { Authorization: `Bearer ${process.env.FISH_API_KEY}`, "Content-Type": "application/json", model: "s2-pro", }; propagation.inject(ctx, headers); const response = await fetch("https://api.fish.audio/v1/tts", { method: "POST", headers, body: JSON.stringify({ text: "Hello from a traced request.", reference_id: "model-id", format: "mp3", }), }); if (!response.ok) { throw new Error(`${response.status} ${await response.text()}`); } }); } finally { span.end(); } ``` ## WebSocket Tracing Pass `traceparent` on the WebSocket upgrade request: ```javascript theme={null} import WebSocket from "ws"; const ws = new WebSocket("wss://api.fish.audio/v1/tts/live", { headers: { Authorization: `Bearer ${process.env.FISH_API_KEY}`, model: "s2-pro", traceparent: "00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01", }, }); ``` Browser WebSocket APIs do not allow custom headers. For browser clients, inject `traceparent` from a trusted server-side proxy or use REST endpoints where your frontend can send the header through `fetch`. ## Enterprise Performance Analysis Fish Audio supports trace-based performance analysis for enterprise customers with a signed enterprise agreement. When this support is enabled, share the W3C trace ID with Fish Audio support so we can correlate your business span with Fish Audio edge routing, backend inference, reference-audio encoding, TTFT, response-time, alignment, and upstream ASR spans. Only share the trace ID or `traceparent` value needed for investigation. Do not place API keys, user identifiers, transcripts, audio URLs, or other sensitive data in trace IDs or span names. # JavaScript SDK Reference Source: https://docs.fish.audio/api-reference/sdk/javascript/api-reference Complete reference for Fish Audio JavaScript SDK ## Client Import and initialize the client: ```typescript theme={null} import { FishAudioClient } from "fish-audio"; const fishAudio = new FishAudioClient({ apiKey: process.env.FISH_API_KEY }); ``` ## Text to Speech ### convert() Generate speech from text. ```typescript theme={null} const audio = await fishAudio.textToSpeech.convert({ text: "Hello" }); ``` Parameters: `request` (TTSRequest), `model?` (Backends)
Returns: `Promise>` ### convertRealtime() Realtime streaming TTS over WebSocket. ```typescript theme={null} async function* textStream() { yield "Hello, "; yield "world!"; } const conn = await fishAudio.textToSpeech.convertRealtime({ text: "" }, textStream()); ``` Parameters: `request` (TTSRequest with `text: ""`), `textStream` (`AsyncIterable`), `backend?` (Backends)
Returns: `RealtimeConnection` (`EventEmitter`-like connection) emitting `RealtimeEvents` ## Speech to Text ### convert() Transcribe audio to text. ```typescript theme={null} const res = await fishAudio.speechToText.convert({ audio: myAudio }); console.log(res.text); ``` Parameters: `request` (STTRequest)
Returns: `STTResponse` ## Voices ### search() List/search available voice models. ```typescript theme={null} const results = await fishAudio.voices.search(); ``` Parameters: `request?` (ModelListRequest)
Returns: `ModelListResponse` ### get() Get model details. ```typescript theme={null} const model = await fishAudio.voices.get("model_id"); ``` Parameters: `voiceId` (string)
Returns: `ModelEntity` ### ivc.create() Create a new voice model from audio samples. ```typescript theme={null} const res = await fishAudio.voices.ivc.create({ title, voices: [file], cover_image: file }); ``` Parameters: `request` (ModelCreateRequest)
Returns: `ModelEntity` ### update() Update model metadata. ```typescript theme={null} await fishAudio.voices.update("model_id", { title: "New Title" }); ``` Parameters: `voiceId` (string), `request` (UpdateModelRequest)
Returns: `UpdateVoiceResponse` ### delete() Delete a model. ```typescript theme={null} await fishAudio.voices.delete("model_id"); ``` Parameters: `voiceId` (string)
Returns: `DeleteVoiceResponse` ## User ### get\_api\_credit() Check API credit balance. ```typescript theme={null} await fishAudio.user.get_api_credit(); ``` Returns: `APICreditResponse` ### get\_package() Get subscription package details. ```typescript theme={null} await fishAudio.user.get_package(); ``` Returns: `PackageResponse` ## Request Classes ### TTSRequest Text-to-speech parameters. ```typescript theme={null} { text: "Hello", reference_id: "model_id", references: [ { audio: File, text: "sample" } ], format: "mp3", prosody: { speed: 1.0, volume: 0 }, } ``` Fields: `text`, `reference_id`, `references`, `format`, `mp3_bitrate`, `opus_bitrate`, `sample_rate`, `prosody`, `latency`, `chunk_length`, `normalize`, `temperature`, `top_p` ### STTRequest Speech-to-text parameters. ```typescript theme={null} { audio: File, language?: "en", ignore_timestamps?: boolean } ``` Fields: `audio`, `language?`, `ignore_timestamps?` ### ReferenceAudio Reference audio for voice cloning. ```typescript theme={null} { audio: File, text: "spoken text" } ``` Fields: `audio`, `text` ### Prosody Speed and volume control. ```typescript theme={null} { speed: 1.2, volume: 5 } ``` Fields: `speed` (0.5–2.0), `volume` (-20 to 20) ### Backends The backend model to use. ```typescript theme={null} Backends = 's1' | 's2-pro'; ``` ## Response Classes ### STTResponse Transcription result. ```typescript theme={null} response.text // Complete transcription response.duration // Duration in seconds response.segments // ASRSegment[] ``` ### ASRSegment Timestamped text segment. Fields: `text` (string), `start` (number, seconds), `end` (number, seconds) ### ModelEntity Voice model information. Fields: `_id`, `title`, `description`, `visibility`, `created_at`, `updated_at`, `tags` ### ModelListResponse List response for voices. Fields: `items` (ModelEntity\[]), `total` (number) ### APICreditResponse API credit information. Fields: `_id` (string), `user_id` (string), `credit` (string), `created_at` (string), `updated_at` (string), `has_phone_sha256` (boolean), `has_free_credit?` (boolean) ### PackageResponse Subscription package details. Fields: `user_id` (string), `type` (string), `total` (number), `balance` (number), `created_at` (string), `updated_at` (string), `finished_at` (string) ## WebSocket Classes ### RealtimeEvents Events emitted by `convertRealtime` connections. | Event | Meaning | | ------------- | ---------------------- | | `OPEN` | Connection established | | `AUDIO_CHUNK` | Audio chunk received | | `ERROR` | Error occurred | | `CLOSE` | Connection closed | ## Event Classes ### StartEvent Stream start event. Fields: `event` ("start"), `request` (TTSRequest) ### TextEvent Text chunk event. Fields: `event` ("text"), `text` (string) ### FlushEvent Flush text chunks event. Fields: `event` ("flush") ### CloseEvent Stream close event. Fields: `event` ("stop") ## Exceptions ### FishAudioError Generic error with status code, body, rawResponse. ### FishAudioTimeoutError Connection timeout error. # Client Source: https://docs.fish.audio/api-reference/sdk/python/client # fishaudio.client Main Fish Audio client classes. ## FishAudio Objects ```python theme={null} class FishAudio() ``` Synchronous Fish Audio API client. **Example**: ```python theme={null} from fishaudio import FishAudio client = FishAudio(api_key="your_api_key") # Generate speech audio = client.tts.convert(text="Hello world") with open("output.mp3", "wb") as f: for chunk in audio: f.write(chunk) # List voices voices = client.voices.list(page_size=20) print(f"Found {voices.total} voices") ``` #### \_\_init\_\_ ```python theme={null} def __init__(*, api_key: Optional[str] = None, base_url: str = "https://api.fish.audio", timeout: float = 240.0, httpx_client: Optional[httpx.Client] = None) ``` Initialize Fish Audio client. **Arguments**: * `api_key` - API key (can also use FISH\_API\_KEY env var) * `base_url` - API base URL * `timeout` - Request timeout in seconds * `httpx_client` - Optional custom HTTP client #### tts ```python theme={null} @property def tts() -> TTSClient ``` Access TTS (text-to-speech) operations. #### asr ```python theme={null} @property def asr() -> ASRClient ``` Access ASR (speech-to-text) operations. #### voices ```python theme={null} @property def voices() -> VoicesClient ``` Access voice management operations. #### account ```python theme={null} @property def account() -> AccountClient ``` Access account/billing operations. #### close ```python theme={null} def close() -> None ``` Close the HTTP client. ## AsyncFishAudio Objects ```python theme={null} class AsyncFishAudio() ``` Asynchronous Fish Audio API client. **Example**: ```python theme={null} from fishaudio import AsyncFishAudio async def main(): client = AsyncFishAudio(api_key="your_api_key") # Generate speech audio = client.tts.convert(text="Hello world") async with aiofiles.open("output.mp3", "wb") as f: async for chunk in audio: await f.write(chunk) # List voices voices = await client.voices.list(page_size=20) print(f"Found {voices.total} voices") asyncio.run(main()) ``` #### \_\_init\_\_ ```python theme={null} def __init__(*, api_key: Optional[str] = None, base_url: str = "https://api.fish.audio", timeout: float = 240.0, httpx_client: Optional[httpx.AsyncClient] = None) ``` Initialize async Fish Audio client. **Arguments**: * `api_key` - API key (can also use FISH\_API\_KEY env var) * `base_url` - API base URL * `timeout` - Request timeout in seconds * `httpx_client` - Optional custom async HTTP client #### tts ```python theme={null} @property def tts() -> AsyncTTSClient ``` Access TTS (text-to-speech) operations. #### asr ```python theme={null} @property def asr() -> AsyncASRClient ``` Access ASR (speech-to-text) operations. #### voices ```python theme={null} @property def voices() -> AsyncVoicesClient ``` Access voice management operations. #### account ```python theme={null} @property def account() -> AsyncAccountClient ``` Access account/billing operations. #### close ```python theme={null} async def close() -> None ``` Close the HTTP client. # Core Source: https://docs.fish.audio/api-reference/sdk/python/core # fishaudio.core.client\_wrapper HTTP client wrapper for managing requests and authentication. ## BaseClientWrapper Objects ```python theme={null} class BaseClientWrapper() ``` Base wrapper with shared logic for sync/async clients. #### get\_headers ```python theme={null} def get_headers( additional_headers: Optional[dict[str, str]] = None) -> dict[str, str] ``` Build headers including authentication and user agent. ## ClientWrapper Objects ```python theme={null} class ClientWrapper(BaseClientWrapper) ``` Wrapper for httpx.Client that handles authentication and error handling. #### request ```python theme={null} def request(method: str, path: str, *, request_options: Optional[RequestOptions] = None, **kwargs: Any) -> httpx.Response ``` Make an HTTP request with error handling. **Arguments**: * `method` - HTTP method (GET, POST, etc.) * `path` - API endpoint path * `request_options` - Optional request-level overrides * `**kwargs` - Additional arguments to pass to httpx.request **Returns**: httpx.Response object **Raises**: * `APIError` - On non-2xx responses #### client ```python theme={null} @property def client() -> httpx.Client ``` Get underlying httpx.Client for advanced usage (e.g., WebSockets). #### close ```python theme={null} def close() -> None ``` Close the HTTP client. ## AsyncClientWrapper Objects ```python theme={null} class AsyncClientWrapper(BaseClientWrapper) ``` Wrapper for httpx.AsyncClient that handles authentication and error handling. #### request ```python theme={null} async def request(method: str, path: str, *, request_options: Optional[RequestOptions] = None, **kwargs: Any) -> httpx.Response ``` Make an async HTTP request with error handling. **Arguments**: * `method` - HTTP method (GET, POST, etc.) * `path` - API endpoint path * `request_options` - Optional request-level overrides * `**kwargs` - Additional arguments to pass to httpx.request **Returns**: httpx.Response object **Raises**: * `APIError` - On non-2xx responses #### client ```python theme={null} @property def client() -> httpx.AsyncClient ``` Get underlying httpx.AsyncClient for advanced usage (e.g., WebSockets). #### close ```python theme={null} async def close() -> None ``` Close the HTTP client. # fishaudio.core.request\_options Request-level options for API calls. ## RequestOptions Objects ```python theme={null} class RequestOptions() ``` Options that can be provided on a per-request basis to override client defaults. **Attributes**: * `timeout` - Override the client's default timeout (in seconds) * `max_retries` - Override the client's default max retries * `additional_headers` - Additional headers to include in the request * `additional_query_params` - Additional query parameters to include #### get\_timeout ```python theme={null} def get_timeout() -> Optional[httpx.Timeout] ``` Convert timeout to httpx.Timeout if set. # fishaudio.core.iterators Audio stream wrappers with collection utilities. ## AudioStream Objects ```python theme={null} class AudioStream() ``` Wrapper for sync audio byte streams with collection utilities. This class wraps an iterator of audio bytes and provides a convenient `.collect()` method to gather all chunks into a single bytes object. **Examples**: ```python theme={null} from fishaudio import FishAudio client = FishAudio(api_key="...") # Collect all audio at once audio = client.tts.stream(text="Hello!").collect() # Or stream chunks manually for chunk in client.tts.stream(text="Hello!"): process_chunk(chunk) ``` #### \_\_init\_\_ ```python theme={null} def __init__(iterator: Iterator[bytes]) ``` Initialize the audio iterator wrapper. **Arguments**: * `iterator` - The underlying iterator of audio bytes #### \_\_iter\_\_ ```python theme={null} def __iter__() -> Iterator[bytes] ``` Allow direct iteration over audio chunks. #### collect ```python theme={null} def collect() -> bytes ``` Collect all audio chunks into a single bytes object. This consumes the iterator and returns all audio data as bytes. After calling this method, the iterator cannot be used again. **Returns**: Complete audio data as bytes **Examples**: ```python theme={null} audio = client.tts.stream(text="Hello!").collect() with open("output.mp3", "wb") as f: f.write(audio) ``` ## AsyncAudioStream Objects ```python theme={null} class AsyncAudioStream() ``` Wrapper for async audio byte streams with collection utilities. This class wraps an async iterator of audio bytes and provides a convenient `.collect()` method to gather all chunks into a single bytes object. **Examples**: ```python theme={null} from fishaudio import AsyncFishAudio client = AsyncFishAudio(api_key="...") # Collect all audio at once stream = await client.tts.stream(text="Hello!") audio = await stream.collect() # Or stream chunks manually async for chunk in await client.tts.stream(text="Hello!"): await process_chunk(chunk) ``` #### \_\_init\_\_ ```python theme={null} def __init__(async_iterator: AsyncIterator[bytes]) ``` Initialize the async audio iterator wrapper. **Arguments**: * `async_iterator` - The underlying async iterator of audio bytes #### \_\_aiter\_\_ ```python theme={null} def __aiter__() -> AsyncIterator[bytes] ``` Allow direct async iteration over audio chunks. #### collect ```python theme={null} async def collect() -> bytes ``` Collect all audio chunks into a single bytes object. This consumes the async iterator and returns all audio data as bytes. After calling this method, the iterator cannot be used again. **Returns**: Complete audio data as bytes **Examples**: ```python theme={null} stream = await client.tts.stream(text="Hello!") audio = await stream.collect() with open("output.mp3", "wb") as f: f.write(audio) ``` # fishaudio.core.websocket\_options WebSocket-level options for WebSocket connections. ## WebSocketOptions Objects ```python theme={null} class WebSocketOptions() ``` Options for configuring WebSocket connections. These options are passed directly to httpx\_ws's connect\_ws/aconnect\_ws functions. For complete documentation, see [https://frankie567.github.io/httpx-ws/reference/httpx\_ws/](https://frankie567.github.io/httpx-ws/reference/httpx_ws/) **Attributes**: * `keepalive_ping_timeout_seconds` - Maximum delay the client will wait for an answer to its Ping event. If the delay is exceeded, WebSocketNetworkError will be raised and the connection closed. Default: 20 seconds. * `keepalive_ping_interval_seconds` - Interval at which the client will automatically send a Ping event to keep the connection alive. Set to None to disable this mechanism. Default: 20 seconds. * `max_message_size_bytes` - Message size in bytes to receive from the server. * `Default` - 65536 bytes (64 KiB). * `queue_size` - Size of the queue where received messages will be held until they are consumed. If the queue is full, the client will stop receiving messages from the server until the queue has room available. Default: 512. **Notes**: Parameter descriptions adapted from httpx\_ws documentation. #### to\_httpx\_ws\_kwargs ```python theme={null} def to_httpx_ws_kwargs() -> dict[str, Any] ``` Convert to kwargs dict for httpx\_ws aconnect\_ws/connect\_ws. # fishaudio.core.omit OMIT sentinel for distinguishing None from not-provided parameters. # Exceptions Source: https://docs.fish.audio/api-reference/sdk/python/exceptions # fishaudio.exceptions Custom exceptions for the Fish Audio SDK. ## FishAudioError Objects ```python theme={null} class FishAudioError(Exception) ``` Base exception for all Fish Audio SDK errors. ## APIError Objects ```python theme={null} class APIError(FishAudioError) ``` Raised when the API returns an error response. ## AuthenticationError Objects ```python theme={null} class AuthenticationError(APIError) ``` Raised when authentication fails (401). ## PermissionError Objects ```python theme={null} class PermissionError(APIError) ``` Raised when permission is denied (403). ## NotFoundError Objects ```python theme={null} class NotFoundError(APIError) ``` Raised when a resource is not found (404). ## RateLimitError Objects ```python theme={null} class RateLimitError(APIError) ``` Raised when rate limit is exceeded (429). ## ServerError Objects ```python theme={null} class ServerError(APIError) ``` Raised when the server encounters an error (5xx). ## WebSocketError Objects ```python theme={null} class WebSocketError(FishAudioError) ``` Raised when WebSocket connection or streaming fails. ## ValidationError Objects ```python theme={null} class ValidationError(FishAudioError) ``` Raised when request validation fails. ## DependencyError Objects ```python theme={null} class DependencyError(FishAudioError) ``` Raised when a required dependency is missing. # Overview Source: https://docs.fish.audio/api-reference/sdk/python/overview Fish Audio Python SDK for text-to-speech and voice cloning ![python.png](https://raw.githubusercontent.com/fishaudio/fish-audio-python/refs/heads/main/.github/assets/python.png) # Fish Audio Python SDK [![PyPI version](https://img.shields.io/pypi/v/fish-audio-sdk.svg)](https://badge.fury.io/py/fish-audio-sdk) [![Python Version](https://img.shields.io/badge/python-3.9+-blue)](https://pypi.org/project/fish-audio-sdk/) [![PyPI - Downloads](https://img.shields.io/pypi/dm/fish-audio-sdk)](https://pypi.org/project/fish-audio-sdk/) [![codecov](https://img.shields.io/codecov/c/github/fishaudio/fish-audio-python)](https://codecov.io/gh/fishaudio/fish-audio-python) [![License](https://img.shields.io/github/license/fishaudio/fish-audio-python)](https://github.com/fishaudio/fish-audio-python/blob/main/LICENSE) The official Python library for the Fish Audio API **Documentation:** [Python SDK Guide](https://docs.fish.audio/developer-guide/sdk-guide/python/) | [API Reference](https://docs.fish.audio/api-reference/sdk/python/) > \[!IMPORTANT] > > ## Changes to PyPI Versioning > > For existing users on Fish Audio Python SDK, please note that the starting version is now `1.0.0`. The last version before this was `2025.6.3`. You may need to adjust your version constraints accordingly. > > The original API in the `fish_audio_sdk` package has NOT been removed, but you will not receive any updates if you continue using the old versioning scheme. > > The simplest fix is to update your dependency to `fish-audio-sdk>=1.0.0` to continue receiving updates, or by pinning to a specific version like `fish-audio-sdk==1.0.0` when installing via your package manager. There are no changes to the API itself in this transition. > > If you're using the legacy `fish_audio_sdk` and would like to switch to the newer, more robust `fishaudio` package, see the [migration guide](https://docs.fish.audio/archive/python-sdk-legacy/migration-guide) to upgrade. ## Installation ```bash theme={null} pip install fish-audio-sdk # With audio playback utilities pip install fish-audio-sdk[utils] ``` ## Authentication Get your API key from [fish.audio/app/api-keys](https://fish.audio/app/api-keys): ```bash theme={null} export FISH_API_KEY=your_api_key_here ``` Or provide directly: ```python theme={null} from fishaudio import FishAudio client = FishAudio(api_key="your_api_key") ``` ## Quick Start **Synchronous:** ```python theme={null} from fishaudio import FishAudio from fishaudio.utils import play, save client = FishAudio() # Generate audio audio = client.tts.convert(text="Hello, world!") # Play or save play(audio) save(audio, "output.mp3") ``` **Asynchronous:** ```python theme={null} import asyncio from fishaudio import AsyncFishAudio from fishaudio.utils import play, save async def main(): client = AsyncFishAudio() audio = await client.tts.convert(text="Hello, world!") play(audio) save(audio, "output.mp3") asyncio.run(main()) ``` ## Core Features ### Text-to-Speech **With custom voice:** ```python theme={null} # Use a specific voice by ID audio = client.tts.convert( text="Custom voice", reference_id="802e3bc2b27e49c2995d23ef70e6ac89" ) ``` **With speed control:** ```python theme={null} audio = client.tts.convert( text="Speaking faster!", speed=1.5 # 1.5x speed ) ``` **Reusable configuration:** ```python theme={null} from fishaudio.types import TTSConfig, Prosody config = TTSConfig( prosody=Prosody(speed=1.2, volume=-5), reference_id="933563129e564b19a115bedd57b7406a", format="wav", latency="balanced" ) # Reuse across generations audio1 = client.tts.convert(text="First message", config=config) audio2 = client.tts.convert(text="Second message", config=config) ``` **Chunk-by-chunk processing:** ```python theme={null} # Stream and process chunks as they arrive for chunk in client.tts.stream(text="Long content..."): send_to_websocket(chunk) # Or collect all chunks audio = client.tts.stream(text="Hello!").collect() ``` [Learn more](https://docs.fish.audio/features/text-to-speech) ### Speech-to-Text ```python theme={null} # Transcribe audio with open("audio.wav", "rb") as f: result = client.asr.transcribe(audio=f.read(), language="en") print(result.text) # Access timestamped segments for segment in result.segments: print(f"[{segment.start:.2f}s - {segment.end:.2f}s] {segment.text}") ``` [Learn more](https://docs.fish.audio/features/speech-to-text) ### Real-time Streaming Stream dynamically generated text for conversational AI and live applications: **Synchronous:** ```python theme={null} def text_chunks(): yield "Hello, " yield "this is " yield "streaming!" audio_stream = client.tts.stream_websocket(text_chunks(), latency="balanced") play(audio_stream) ``` **Asynchronous:** ```python theme={null} import asyncio from fishaudio import AsyncFishAudio async def text_chunks(): yield "Hello, " yield "this is " yield "streaming!" async def main(): async with AsyncFishAudio() as client: # stream_websocket is an async generator — iterate it, don't await the call audio_stream = client.tts.stream_websocket(text_chunks(), latency="balanced") with open("out.mp3", "wb") as f: async for chunk in audio_stream: f.write(chunk) asyncio.run(main()) ``` [Learn more](https://docs.fish.audio/features/realtime-streaming) ### Voice Cloning **Instant cloning:** ```python theme={null} from fishaudio.types import ReferenceAudio # Clone voice on-the-fly with open("reference.wav", "rb") as f: audio = client.tts.convert( text="Cloned voice speaking", references=[ReferenceAudio( audio=f.read(), text="Text spoken in reference" )] ) ``` **Persistent voice models:** ```python theme={null} # Create voice model for reuse with open("voice_sample.wav", "rb") as f: voice = client.voices.create( title="My Voice", voices=[f.read()], description="Custom voice clone" ) # Use the created model audio = client.tts.convert( text="Using my saved voice", reference_id=voice.id ) ``` [Learn more](https://docs.fish.audio/features/voice-cloning) ## Resource Clients | Resource | Description | Key Methods | | ---------------- | ------------------ | ----------------------------------------------------- | | `client.tts` | Text-to-speech | `convert()`, `stream()`, `stream_websocket()` | | `client.asr` | Speech recognition | `transcribe()` | | `client.voices` | Voice management | `list()`, `get()`, `create()`, `update()`, `delete()` | | `client.account` | Account info | `get_credits()`, `get_package()` | ## Error Handling ```python theme={null} from fishaudio.exceptions import ( AuthenticationError, RateLimitError, NotFoundError, APIError, FishAudioError, ) try: audio = client.tts.convert(text="Hello!") except AuthenticationError: print("Invalid API key") except RateLimitError: print("Rate limit exceeded") except NotFoundError: print("Voice model not found") except APIError as e: print(f"API error {e.status}: {e.message}") # any other HTTP error, including 422 validation except FishAudioError as e: print(f"SDK error: {e}") ``` ## Resources * **Documentation:** [SDK Guide](https://docs.fish.audio/developer-guide/sdk-guide/python/) | [API Reference](https://docs.fish.audio/api-reference/sdk/python/) * **Package:** [PyPI](https://pypi.org/project/fish-audio-sdk/) | [GitHub](https://github.com/fishaudio/fish-audio-python) * **Legacy SDK:** [Documentation](https://docs.fish.audio/archive/python-sdk-legacy) | [Migration Guide](https://docs.fish.audio/archive/python-sdk-legacy/migration-guide) ## License This project is licensed under the Apache-2.0 License - see the [LICENSE](LICENSE) file for details. # Resources Source: https://docs.fish.audio/api-reference/sdk/python/resources # fishaudio.resources.voices Voice management namespace client. ## VoicesClient Objects ```python theme={null} class VoicesClient() ``` Synchronous voice management operations. #### list ```python theme={null} def list( *, page_size: int = 10, page_number: int = 1, title: Optional[str] = OMIT, tags: Optional[Union[list[str], str]] = OMIT, self_only: bool = False, author_id: Optional[str] = OMIT, language: Optional[Union[list[str], str]] = OMIT, title_language: Optional[Union[list[str], str]] = OMIT, sort_by: str = "task_count", request_options: Optional[RequestOptions] = None ) -> PaginatedResponse[Voice] ``` List available voices/models. **Arguments**: * `page_size` - Number of results per page * `page_number` - Page number (1-indexed) * `title` - Filter by title * `tags` - Filter by tags (single tag or list) * `self_only` - Only return user's own voices * `author_id` - Filter by author ID * `language` - Filter by language(s) * `title_language` - Filter by title language(s) * `sort_by` - Sort field ("task\_count" or "created\_at") * `request_options` - Request-level overrides **Returns**: Paginated response with total count and voice items **Example**: ```python theme={null} client = FishAudio(api_key="...") # List all voices voices = client.voices.list(page_size=20) print(f"Total: {voices.total}") for voice in voices.items: print(f"{voice.title}: {voice.id}") # Filter by tags tagged = client.voices.list(tags=["male", "english"]) ``` #### get ```python theme={null} def get(voice_id: str, *, request_options: Optional[RequestOptions] = None) -> Voice ``` Get voice by ID. **Arguments**: * `voice_id` - Voice model ID * `request_options` - Request-level overrides **Returns**: Voice model details **Example**: ```python theme={null} client = FishAudio(api_key="...") voice = client.voices.get("voice_id_here") print(voice.title, voice.description) ``` #### create ```python theme={null} def create(*, title: str, voices: builtins.list[bytes], description: Optional[str] = OMIT, texts: Optional[builtins.list[str]] = OMIT, tags: Optional[builtins.list[str]] = OMIT, cover_image: Optional[bytes] = OMIT, visibility: Visibility = "private", train_mode: str = "fast", enhance_audio_quality: bool = True, request_options: Optional[RequestOptions] = None) -> Voice ``` Create/clone a new voice. **Arguments**: * `title` - Voice model name * `voices` - List of audio file bytes for training * `description` - Voice description * `texts` - Transcripts for voice samples * `tags` - Tags for categorization * `cover_image` - Cover image bytes * `visibility` - Visibility setting (public, unlist, private) * `train_mode` - Training mode (currently only "fast" supported) * `enhance_audio_quality` - Whether to enhance audio quality * `request_options` - Request-level overrides **Returns**: Created voice model **Example**: ```python theme={null} client = FishAudio(api_key="...") with open("voice1.wav", "rb") as f1, open("voice2.wav", "rb") as f2: voice = client.voices.create( title="My Voice", voices=[f1.read(), f2.read()], description="Custom voice clone", tags=["custom", "english"] ) print(f"Created: {voice.id}") ``` #### update ```python theme={null} def update(voice_id: str, *, title: Optional[str] = OMIT, description: Optional[str] = OMIT, cover_image: Optional[bytes] = OMIT, visibility: Optional[Visibility] = OMIT, tags: Optional[builtins.list[str]] = OMIT, request_options: Optional[RequestOptions] = None) -> None ``` Update voice metadata. **Arguments**: * `voice_id` - Voice model ID * `title` - New title * `description` - New description * `cover_image` - New cover image bytes * `visibility` - New visibility setting * `tags` - New tags * `request_options` - Request-level overrides **Example**: ```python theme={null} client = FishAudio(api_key="...") client.voices.update( "voice_id_here", title="Updated Title", visibility="public" ) ``` #### delete ```python theme={null} def delete(voice_id: str, *, request_options: Optional[RequestOptions] = None) -> None ``` Delete a voice. **Arguments**: * `voice_id` - Voice model ID * `request_options` - Request-level overrides **Example**: ```python theme={null} client = FishAudio(api_key="...") client.voices.delete("voice_id_here") ``` ## AsyncVoicesClient Objects ```python theme={null} class AsyncVoicesClient() ``` Asynchronous voice management operations. #### list ```python theme={null} async def list( *, page_size: int = 10, page_number: int = 1, title: Optional[str] = OMIT, tags: Optional[Union[list[str], str]] = OMIT, self_only: bool = False, author_id: Optional[str] = OMIT, language: Optional[Union[list[str], str]] = OMIT, title_language: Optional[Union[list[str], str]] = OMIT, sort_by: str = "task_count", request_options: Optional[RequestOptions] = None ) -> PaginatedResponse[Voice] ``` List available voices/models (async). See sync version for details. #### get ```python theme={null} async def get(voice_id: str, *, request_options: Optional[RequestOptions] = None) -> Voice ``` Get voice by ID (async). See sync version for details. #### create ```python theme={null} async def create(*, title: str, voices: builtins.list[bytes], description: Optional[str] = OMIT, texts: Optional[builtins.list[str]] = OMIT, tags: Optional[builtins.list[str]] = OMIT, cover_image: Optional[bytes] = OMIT, visibility: Visibility = "private", train_mode: str = "fast", enhance_audio_quality: bool = True, request_options: Optional[RequestOptions] = None) -> Voice ``` Create/clone a new voice (async). See sync version for details. #### update ```python theme={null} async def update(voice_id: str, *, title: Optional[str] = OMIT, description: Optional[str] = OMIT, cover_image: Optional[bytes] = OMIT, visibility: Optional[Visibility] = OMIT, tags: Optional[builtins.list[str]] = OMIT, request_options: Optional[RequestOptions] = None) -> None ``` Update voice metadata (async). See sync version for details. #### delete ```python theme={null} async def delete(voice_id: str, *, request_options: Optional[RequestOptions] = None) -> None ``` Delete a voice (async). See sync version for details. # fishaudio.resources.account Account namespace client for billing and credits. ## AccountClient Objects ```python theme={null} class AccountClient() ``` Synchronous account operations. #### get\_credits ```python theme={null} def get_credits(*, check_free_credit: Optional[bool] = OMIT, request_options: Optional[RequestOptions] = None) -> Credits ``` Get API credit balance. **Arguments**: * `check_free_credit` - Whether to check free credit availability * `request_options` - Request-level overrides **Returns**: Credits information **Example**: ```python theme={null} client = FishAudio(api_key="...") credits = client.account.get_credits() print(f"Available credits: {float(credits.credit)}") # Check free credit availability credits = client.account.get_credits(check_free_credit=True) if credits.has_free_credit: print("Free credits available!") ``` #### get\_package ```python theme={null} def get_package(*, request_options: Optional[RequestOptions] = None) -> Package ``` Get package information. **Arguments**: * `request_options` - Request-level overrides **Returns**: Package information **Example**: ```python theme={null} client = FishAudio(api_key="...") package = client.account.get_package() print(f"Balance: {package.balance}/{package.total}") ``` ## AsyncAccountClient Objects ```python theme={null} class AsyncAccountClient() ``` Asynchronous account operations. #### get\_credits ```python theme={null} async def get_credits( *, check_free_credit: Optional[bool] = OMIT, request_options: Optional[RequestOptions] = None) -> Credits ``` Get API credit balance (async). **Arguments**: * `check_free_credit` - Whether to check free credit availability * `request_options` - Request-level overrides **Returns**: Credits information **Example**: ```python theme={null} client = AsyncFishAudio(api_key="...") credits = await client.account.get_credits() print(f"Available credits: {float(credits.credit)}") # Check free credit availability credits = await client.account.get_credits(check_free_credit=True) if credits.has_free_credit: print("Free credits available!") ``` #### get\_package ```python theme={null} async def get_package(*, request_options: Optional[RequestOptions] = None ) -> Package ``` Get package information (async). **Arguments**: * `request_options` - Request-level overrides **Returns**: Package information **Example**: ```python theme={null} client = AsyncFishAudio(api_key="...") package = await client.account.get_package() print(f"Balance: {package.balance}/{package.total}") ``` # fishaudio.resources.tts TTS (Text-to-Speech) namespace client. ## TTSClient Objects ```python theme={null} class TTSClient() ``` Synchronous TTS operations. #### stream ```python theme={null} def stream(*, text: str, reference_id: Optional[str] = None, references: Optional[list[ReferenceAudio]] = None, format: Optional[AudioFormat] = None, latency: Optional[LatencyMode] = None, speed: Optional[float] = None, config: TTSConfig = TTSConfig(), model: Model = "s2-pro", request_options: Optional[RequestOptions] = None) -> AudioStream ``` Stream text-to-speech audio chunks. **Arguments**: * `text` - Text to synthesize * `reference_id` - Voice reference ID (overrides config.reference\_id if provided) * `references` - Reference audio samples (overrides config.references if provided) * `format` - Audio format - "mp3", "wav", "pcm", or "opus" (overrides config.format if provided) * `latency` - Latency mode - "normal" or "balanced" (overrides config.latency if provided) * `speed` - Speech speed multiplier, e.g. 1.5 for 1.5x speed (overrides config.prosody.speed if provided) * `config` - TTS configuration (audio settings, voice, model parameters) * `model` - TTS model to use * `request_options` - Request-level overrides **Returns**: AudioStream object that can be iterated for audio chunks **Example**: ```python theme={null} from fishaudio import FishAudio client = FishAudio(api_key="...") # Stream and process chunks for chunk in client.tts.stream(text="Hello world"): process_audio_chunk(chunk) # Or collect all at once audio = client.tts.stream(text="Hello world").collect() ``` #### convert ```python theme={null} def convert(*, text: str, reference_id: Optional[str] = None, references: Optional[list[ReferenceAudio]] = None, format: Optional[AudioFormat] = None, latency: Optional[LatencyMode] = None, speed: Optional[float] = None, config: TTSConfig = TTSConfig(), model: Model = "s2-pro", request_options: Optional[RequestOptions] = None) -> bytes ``` Convert text to speech and return complete audio as bytes. This is a convenience method that streams all audio chunks and combines them. For chunk-by-chunk processing, use stream() instead. **Arguments**: * `text` - Text to synthesize * `reference_id` - Voice reference ID (overrides config.reference\_id if provided) * `references` - Reference audio samples (overrides config.references if provided) * `format` - Audio format - "mp3", "wav", "pcm", or "opus" (overrides config.format if provided) * `latency` - Latency mode - "normal" or "balanced" (overrides config.latency if provided) * `speed` - Speech speed multiplier, e.g. 1.5 for 1.5x speed (overrides config.prosody.speed if provided) * `config` - TTS configuration (audio settings, voice, model parameters) * `model` - TTS model to use * `request_options` - Request-level overrides **Returns**: Complete audio as bytes **Example**: ```python theme={null} from fishaudio import FishAudio from fishaudio.utils import play, save client = FishAudio(api_key="...") # Get complete audio audio = client.tts.convert(text="Hello world") # Play it play(audio) # Or save it save(audio, "output.mp3") ``` #### stream\_websocket ```python theme={null} def stream_websocket( text_stream: Iterable[Union[str, TextEvent, FlushEvent]], *, reference_id: Optional[str] = None, references: Optional[list[ReferenceAudio]] = None, format: Optional[AudioFormat] = None, latency: Optional[LatencyMode] = None, speed: Optional[float] = None, config: TTSConfig = TTSConfig(), model: Model = "s2-pro", max_workers: int = 10, ws_options: Optional[WebSocketOptions] = None) -> Iterator[bytes] ``` Stream text and receive audio in real-time via WebSocket. Perfect for conversational AI, live captioning, and streaming applications. **Arguments**: * `text_stream` - Iterator of text chunks to stream * `reference_id` - Voice reference ID (overrides config.reference\_id if provided) * `references` - Reference audio samples (overrides config.references if provided) * `format` - Audio format - "mp3", "wav", "pcm", or "opus" (overrides config.format if provided) * `latency` - Latency mode - "normal" or "balanced" (overrides config.latency if provided) * `speed` - Speech speed multiplier, e.g. 1.5 for 1.5x speed (overrides config.prosody.speed if provided) * `config` - TTS configuration (audio settings, voice, model parameters) * `model` - TTS model to use * `max_workers` - ThreadPoolExecutor workers for concurrent sender * `ws_options` - WebSocket connection options for configuring timeouts, message size limits, etc. Useful for long-running generations that may exceed default timeout values. See WebSocketOptions class for available parameters. **Returns**: Iterator of audio bytes **Example**: ```python theme={null} from fishaudio import FishAudio, TTSConfig, ReferenceAudio, WebSocketOptions client = FishAudio(api_key="...") def text_generator(): yield "Hello, " yield "this is " yield "streaming text!" # Simple usage with defaults with open("output.mp3", "wb") as f: for audio_chunk in client.tts.stream_websocket(text_generator()): f.write(audio_chunk) # With format and speed parameters with open("output.wav", "wb") as f: for audio_chunk in client.tts.stream_websocket( text_generator(), format="wav", speed=1.3 ): f.write(audio_chunk) # With reference_id parameter with open("output.mp3", "wb") as f: for audio_chunk in client.tts.stream_websocket(text_generator(), reference_id="your_model_id"): f.write(audio_chunk) # With references parameter with open("output.mp3", "wb") as f: for audio_chunk in client.tts.stream_websocket( text_generator(), references=[ReferenceAudio(audio=audio_bytes, text="sample")] ): f.write(audio_chunk) # With WebSocket options for long-running generations # Useful if you're generating very long responses that may take >20 seconds ws_options = WebSocketOptions(keepalive_ping_timeout_seconds=60.0) with open("output.mp3", "wb") as f: for audio_chunk in client.tts.stream_websocket( text_generator(), ws_options=ws_options ): f.write(audio_chunk) # Parameters override config values config = TTSConfig(format="mp3", latency="balanced") with open("output.wav", "wb") as f: for audio_chunk in client.tts.stream_websocket( text_generator(), format="wav", # Parameter wins config=config ): f.write(audio_chunk) ``` ## AsyncTTSClient Objects ```python theme={null} class AsyncTTSClient() ``` Asynchronous TTS operations. #### stream ```python theme={null} async def stream( *, text: str, reference_id: Optional[str] = None, references: Optional[list[ReferenceAudio]] = None, format: Optional[AudioFormat] = None, latency: Optional[LatencyMode] = None, speed: Optional[float] = None, config: TTSConfig = TTSConfig(), model: Model = "s2-pro", request_options: Optional[RequestOptions] = None) -> AsyncAudioStream ``` Stream text-to-speech audio chunks (async). **Arguments**: * `text` - Text to synthesize * `reference_id` - Voice reference ID (overrides config.reference\_id if provided) * `references` - Reference audio samples (overrides config.references if provided) * `format` - Audio format - "mp3", "wav", "pcm", or "opus" (overrides config.format if provided) * `latency` - Latency mode - "normal" or "balanced" (overrides config.latency if provided) * `speed` - Speech speed multiplier, e.g. 1.5 for 1.5x speed (overrides config.prosody.speed if provided) * `config` - TTS configuration (audio settings, voice, model parameters) * `model` - TTS model to use * `request_options` - Request-level overrides **Returns**: AsyncAudioStream object that can be iterated for audio chunks **Example**: ```python theme={null} from fishaudio import AsyncFishAudio client = AsyncFishAudio(api_key="...") # Stream and process chunks async for chunk in await client.tts.stream(text="Hello world"): await process_audio_chunk(chunk) # Or collect all at once stream = await client.tts.stream(text="Hello world") audio = await stream.collect() ``` #### convert ```python theme={null} async def convert(*, text: str, reference_id: Optional[str] = None, references: Optional[list[ReferenceAudio]] = None, format: Optional[AudioFormat] = None, latency: Optional[LatencyMode] = None, speed: Optional[float] = None, config: TTSConfig = TTSConfig(), model: Model = "s2-pro", request_options: Optional[RequestOptions] = None) -> bytes ``` Convert text to speech and return complete audio as bytes (async). This is a convenience method that streams all audio chunks and combines them. For chunk-by-chunk processing, use stream() instead. **Arguments**: * `text` - Text to synthesize * `reference_id` - Voice reference ID (overrides config.reference\_id if provided) * `references` - Reference audio samples (overrides config.references if provided) * `format` - Audio format - "mp3", "wav", "pcm", or "opus" (overrides config.format if provided) * `latency` - Latency mode - "normal" or "balanced" (overrides config.latency if provided) * `speed` - Speech speed multiplier, e.g. 1.5 for 1.5x speed (overrides config.prosody.speed if provided) * `config` - TTS configuration (audio settings, voice, model parameters) * `model` - TTS model to use * `request_options` - Request-level overrides **Returns**: Complete audio as bytes **Example**: ```python theme={null} from fishaudio import AsyncFishAudio from fishaudio.utils import play, save client = AsyncFishAudio(api_key="...") # Get complete audio audio = await client.tts.convert(text="Hello world") # Play it play(audio) # Or save it save(audio, "output.mp3") ``` #### stream\_websocket ```python theme={null} async def stream_websocket(text_stream: AsyncIterable[Union[str, TextEvent, FlushEvent]], *, reference_id: Optional[str] = None, references: Optional[list[ReferenceAudio]] = None, format: Optional[AudioFormat] = None, latency: Optional[LatencyMode] = None, speed: Optional[float] = None, config: TTSConfig = TTSConfig(), model: Model = "s2-pro", ws_options: Optional[WebSocketOptions] = None) ``` Stream text and receive audio in real-time via WebSocket (async). Perfect for conversational AI, live captioning, and streaming applications. **Arguments**: * `text_stream` - Async iterator of text chunks to stream * `reference_id` - Voice reference ID (overrides config.reference\_id if provided) * `references` - Reference audio samples (overrides config.references if provided) * `format` - Audio format - "mp3", "wav", "pcm", or "opus" (overrides config.format if provided) * `latency` - Latency mode - "normal" or "balanced" (overrides config.latency if provided) * `speed` - Speech speed multiplier, e.g. 1.5 for 1.5x speed (overrides config.prosody.speed if provided) * `config` - TTS configuration (audio settings, voice, model parameters) * `model` - TTS model to use * `ws_options` - WebSocket connection options for configuring timeouts, message size limits, etc. Useful for long-running generations that may exceed default timeout values. See WebSocketOptions class for available parameters. **Returns**: Async iterator of audio bytes **Example**: ```python theme={null} from fishaudio import AsyncFishAudio, TTSConfig, ReferenceAudio, WebSocketOptions client = AsyncFishAudio(api_key="...") async def text_generator(): yield "Hello, " yield "this is " yield "async streaming!" # Simple usage with defaults async with aiofiles.open("output.mp3", "wb") as f: async for audio_chunk in client.tts.stream_websocket(text_generator()): await f.write(audio_chunk) # With format and speed parameters async with aiofiles.open("output.wav", "wb") as f: async for audio_chunk in client.tts.stream_websocket( text_generator(), format="wav", speed=1.3 ): await f.write(audio_chunk) # With reference_id parameter async with aiofiles.open("output.mp3", "wb") as f: async for audio_chunk in client.tts.stream_websocket(text_generator(), reference_id="your_model_id"): await f.write(audio_chunk) # With references parameter async with aiofiles.open("output.mp3", "wb") as f: async for audio_chunk in client.tts.stream_websocket( text_generator(), references=[ReferenceAudio(audio=audio_bytes, text="sample")] ): await f.write(audio_chunk) # With WebSocket options for long-running generations # Useful if you're generating very long responses that may take >20 seconds ws_options = WebSocketOptions(keepalive_ping_timeout_seconds=60.0) async with aiofiles.open("output.mp3", "wb") as f: async for audio_chunk in client.tts.stream_websocket( text_generator(), ws_options=ws_options ): await f.write(audio_chunk) # Parameters override config values config = TTSConfig(format="mp3", latency="balanced") async with aiofiles.open("output.wav", "wb") as f: async for audio_chunk in client.tts.stream_websocket( text_generator(), format="wav", # Parameter wins config=config ): await f.write(audio_chunk) ``` # fishaudio.resources.realtime Real-time WebSocket streaming helpers. #### iter\_websocket\_audio ```python theme={null} def iter_websocket_audio(ws) -> Iterator[bytes] ``` Process WebSocket audio messages (sync). Receives messages from WebSocket, yields audio chunks, handles errors. Unknown events are ignored and iteration continues. **Arguments**: * `ws` - WebSocket connection from httpx\_ws.connect\_ws **Yields**: Audio bytes **Raises**: * `WebSocketError` - On disconnect or error finish event #### aiter\_websocket\_audio ```python theme={null} async def aiter_websocket_audio(ws) -> AsyncIterator[bytes] ``` Process WebSocket audio messages (async). Receives messages from WebSocket, yields audio chunks, handles errors. Unknown events are ignored and iteration continues. **Arguments**: * `ws` - WebSocket connection from httpx\_ws.aconnect\_ws **Yields**: Audio bytes **Raises**: * `WebSocketError` - On disconnect or error finish event # fishaudio.resources.asr ASR (Automatic Speech Recognition) namespace client. ## ASRClient Objects ```python theme={null} class ASRClient() ``` Synchronous ASR operations. #### transcribe ```python theme={null} def transcribe( *, audio: bytes, language: Optional[str] = OMIT, include_timestamps: bool = True, request_options: Optional[RequestOptions] = None) -> ASRResponse ``` Transcribe audio to text. **Arguments**: * `audio` - Audio file bytes * `language` - Language code (e.g., "en", "zh"). Auto-detected if not provided. * `include_timestamps` - Whether to include timestamp information for segments * `request_options` - Request-level overrides **Returns**: ASRResponse with transcription text, duration, and segments **Example**: ```python theme={null} client = FishAudio(api_key="...") with open("audio.mp3", "rb") as f: audio_bytes = f.read() result = client.asr.transcribe(audio=audio_bytes, language="en") print(result.text) for segment in result.segments: print(f"{segment.start}-{segment.end}: {segment.text}") ``` ## AsyncASRClient Objects ```python theme={null} class AsyncASRClient() ``` Asynchronous ASR operations. #### transcribe ```python theme={null} async def transcribe( *, audio: bytes, language: Optional[str] = OMIT, include_timestamps: bool = True, request_options: Optional[RequestOptions] = None) -> ASRResponse ``` Transcribe audio to text (async). **Arguments**: * `audio` - Audio file bytes * `language` - Language code (e.g., "en", "zh"). Auto-detected if not provided. * `include_timestamps` - Whether to include timestamp information for segments * `request_options` - Request-level overrides **Returns**: ASRResponse with transcription text, duration, and segments **Example**: ```python theme={null} client = AsyncFishAudio(api_key="...") async with aiofiles.open("audio.mp3", "rb") as f: audio_bytes = await f.read() result = await client.asr.transcribe(audio=audio_bytes, language="en") print(result.text) for segment in result.segments: print(f"{segment.start}-{segment.end}: {segment.text}") ``` # Types Source: https://docs.fish.audio/api-reference/sdk/python/types # fishaudio.types.voices Voice and model management types. ## Sample Objects ```python theme={null} class Sample(BaseModel) ``` A sample audio for a voice model. **Attributes**: * `title` - Title/name of the audio sample * `text` - Transcription of the spoken content in the sample * `task_id` - Unique identifier for the sample task * `audio` - URL or path to the audio file ## Author Objects ```python theme={null} class Author(BaseModel) ``` Voice model author information. **Attributes**: * `id` - Unique author identifier * `nickname` - Author's display name * `avatar` - URL to author's avatar image ## Voice Objects ```python theme={null} class Voice(BaseModel) ``` A voice model. Represents a TTS voice that can be used for synthesis. **Attributes**: * `id` - Unique voice model identifier (use as reference\_id in TTS) * `type` - Model type. Options: "svc" (singing voice conversion), "tts" (text-to-speech) * `title` - Voice model title/name * `description` - Detailed description of the voice model * `cover_image` - URL to the voice model's cover image * `train_mode` - Training mode used. Options: "fast" * `state` - Current model state: "created", "training", "trained", or "failed" * `tags` - List of tags for categorization (e.g., \["male", "english", "young"]) * `samples` - List of audio samples demonstrating the voice * `created_at` - Timestamp when the model was created * `updated_at` - Timestamp when the model was last updated * `languages` - List of supported language codes (e.g., \["en", "zh"]) * `visibility` - Model visibility. Options: "public", "private", "unlist" * `lock_visibility` - Whether visibility setting is locked * `like_count` - Number of likes the model has received * `mark_count` - Number of bookmarks/favorites * `shared_count` - Number of times the model has been shared * `task_count` - Number of times the model has been used for generation * `liked` - Whether the current user has liked this model. Default: False * `marked` - Whether the current user has bookmarked this model. Default: False * `author` - Information about the voice model's creator # fishaudio.types.account Account-related types (credits, packages, etc.). ## Credits Objects ```python theme={null} class Credits(BaseModel) ``` User's API credit balance. **Attributes**: * `id` - Unique credits record identifier * `user_id` - User identifier * `credit` - Current credit balance (decimal for precise accounting) * `created_at` - Timestamp when the credits record was created * `updated_at` - Timestamp when the credits were last updated * `has_phone_sha256` - Whether the user has a verified phone number. Optional * `has_free_credit` - Whether the user has received free credits. Optional ## Package Objects ```python theme={null} class Package(BaseModel) ``` User's prepaid package information. **Attributes**: * `id` - Unique package identifier * `user_id` - User identifier * `type` - Package type identifier * `total` - Total units in the package * `balance` - Remaining units in the package * `created_at` - Timestamp when the package was purchased * `updated_at` - Timestamp when the package was last updated * `finished_at` - Timestamp when the package was fully consumed. None if still active # fishaudio.types.tts TTS-related types. ## ReferenceAudio Objects ```python theme={null} class ReferenceAudio(BaseModel) ``` Reference audio for voice cloning/style. **Attributes**: * `audio` - Audio file bytes for the reference sample * `text` - Transcription of what is spoken in the reference audio. Should match exactly what's spoken and include punctuation for proper prosody. ## Prosody Objects ```python theme={null} class Prosody(BaseModel) ``` Speech prosody settings (speed and volume). **Attributes**: * `speed` - Speech speed multiplier. Range: 0.5-2.0. Default: 1.0. * `Examples` - 1.5 = 50% faster, 0.8 = 20% slower * `volume` - Volume adjustment in decibels. Range: -20.0 to 20.0. Default: 0.0 (no change). Positive values increase volume, negative values decrease it. #### from\_speed\_override ```python theme={null} @classmethod def from_speed_override(cls, speed: float, base: Optional["Prosody"] = None) -> "Prosody" ``` Create Prosody with speed override, preserving volume from base. **Arguments**: * `speed` - Speed value to use * `base` - Base prosody to preserve volume from (if any) **Returns**: New Prosody instance with overridden speed ## TTSConfig Objects ```python theme={null} class TTSConfig(BaseModel) ``` TTS generation configuration. Reusable configuration for text-to-speech requests. Create once, use multiple times. All parameters have sensible defaults. **Attributes**: * `format` - Audio output format. Options: "mp3", "wav", "pcm", "opus". Default: "mp3" * `sample_rate` - Audio sample rate in Hz. If None, uses format-specific default. * `mp3_bitrate` - MP3 bitrate in kbps. Options: 64, 128, 192. Default: 128 * `opus_bitrate` - Opus bitrate in kbps. Options: -1000, 24, 32, 48, 64. Default: 32 * `normalize` - Whether to normalize/clean the input text. Default: True * `chunk_length` - Characters per generation chunk. Range: 100-300. Default: 200. Lower values = faster initial response, higher values = better quality * `latency` - Generation mode. Options: "normal" (higher quality), "balanced" (faster). Default: "balanced" * `reference_id` - Voice model ID from fish.audio (e.g., "802e3bc2b27e49c2995d23ef70e6ac89"). Find IDs in voice URLs or via voices.list() * `references` - List of reference audio samples for instant voice cloning. Default: \[] * `prosody` - Speech speed and volume settings. Default: None (uses natural prosody) * `top_p` - Nucleus sampling parameter for token selection. Range: 0.0-1.0. Default: 0.7 * `temperature` - Randomness in generation. Range: 0.0-1.0. Default: 0.7. Higher = more varied, lower = more consistent * `max_new_tokens` - Maximum number of tokens to generate. Default: 1024 * `repetition_penalty` - Penalty for repeated tokens. Default: 1.2 * `min_chunk_length` - Minimum chunk length for generation. Default: 50 * `condition_on_previous_chunks` - Whether to condition generation on previous chunks. Default: True * `early_stop_threshold` - Threshold for early stopping. Default: 1.0 ## TTSRequest Objects ```python theme={null} class TTSRequest(BaseModel) ``` Request parameters for text-to-speech generation. This model is used internally for WebSocket streaming. For the HTTP API, parameters are passed directly to methods. **Attributes**: * `text` - Text to synthesize into speech * `chunk_length` - Characters per generation chunk. Range: 100-300. Default: 200 * `format` - Audio output format. Options: "mp3", "wav", "pcm", "opus". Default: "mp3" * `sample_rate` - Audio sample rate in Hz. If None, uses format-specific default * `mp3_bitrate` - MP3 bitrate in kbps. Options: 64, 128, 192. Default: 128 * `opus_bitrate` - Opus bitrate in kbps. Options: -1000, 24, 32, 48, 64. Default: 32 * `references` - List of reference audio samples for voice cloning. Default: \[] * `reference_id` - Voice model ID for using a specific voice. Default: None * `normalize` - Whether to normalize/clean the input text. Default: True * `latency` - Generation mode. Options: "normal", "balanced". Default: "balanced" * `prosody` - Speech speed and volume settings. Default: None * `top_p` - Nucleus sampling for token selection. Range: 0.0-1.0. Default: 0.7 * `temperature` - Randomness in generation. Range: 0.0-1.0. Default: 0.7 * `max_new_tokens` - Maximum number of tokens to generate. Default: 1024 * `repetition_penalty` - Penalty for repeated tokens. Default: 1.2 * `min_chunk_length` - Minimum chunk length for generation. Default: 50 * `condition_on_previous_chunks` - Whether to condition generation on previous chunks. Default: True * `early_stop_threshold` - Threshold for early stopping. Default: 1.0 ## StartEvent Objects ```python theme={null} class StartEvent(BaseModel) ``` WebSocket start event to initiate TTS streaming. **Attributes**: * `event` - Event type identifier, always "start" * `request` - TTS configuration for the streaming session ## TextEvent Objects ```python theme={null} class TextEvent(BaseModel) ``` WebSocket event to send a text chunk for synthesis. **Attributes**: * `event` - Event type identifier, always "text" * `text` - Text chunk to synthesize ## FlushEvent Objects ```python theme={null} class FlushEvent(BaseModel) ``` WebSocket event to force immediate audio generation from buffered text. Use this to ensure all buffered text is synthesized without waiting for more input. **Attributes**: * `event` - Event type identifier, always "flush" ## CloseEvent Objects ```python theme={null} class CloseEvent(BaseModel) ``` WebSocket event to end the streaming session. **Attributes**: * `event` - Event type identifier, always "stop" # fishaudio.types.shared Shared types used across the SDK. ## PaginatedResponse Objects ```python theme={null} class PaginatedResponse(BaseModel, Generic[T]) ``` Generic paginated response. **Attributes**: * `total` - Total number of items across all pages * `items` - List of items on the current page #### warn\_if\_deprecated\_model ```python theme={null} def warn_if_deprecated_model(model: str) -> None ``` Emit a deprecation warning if a legacy model is used. # fishaudio.types.asr ASR (Automatic Speech Recognition) related types. ## ASRSegment Objects ```python theme={null} class ASRSegment(BaseModel) ``` A timestamped segment of transcribed text. **Attributes**: * `text` - The transcribed text for this segment * `start` - Segment start time in seconds * `end` - Segment end time in seconds ## ASRResponse Objects ```python theme={null} class ASRResponse(BaseModel) ``` Response from speech-to-text transcription. **Attributes**: * `text` - Complete transcription of the entire audio * `duration` - Total audio duration in seconds * `segments` - List of timestamped text segments. Empty if include\_timestamps=False #### duration Duration in seconds # Utils Source: https://docs.fish.audio/api-reference/sdk/python/utils # fishaudio.utils.play Audio playback utility. #### play ```python theme={null} def play(audio: Union[bytes, Iterable[bytes]], *, notebook: bool = False, use_ffmpeg: bool = True) -> None ``` Play audio using various playback methods. **Arguments**: * `audio` - Audio bytes or iterable of bytes * `notebook` - Use Jupyter notebook playback (IPython.display.Audio) * `use_ffmpeg` - Use ffplay for playback (default, falls back to sounddevice) **Raises**: * `DependencyError` - If required playback tool is not installed **Examples**: ```python theme={null} from fishaudio import FishAudio, play client = FishAudio(api_key="...") audio = client.tts.convert(text="Hello world") # Play directly play(audio) # In Jupyter notebook play(audio, notebook=True) # Force sounddevice fallback play(audio, use_ffmpeg=False) ``` # fishaudio.utils.save Audio saving utility. #### save ```python theme={null} def save(audio: Union[bytes, Iterable[bytes]], filename: str) -> None ``` Save audio to a file. **Arguments**: * `audio` - Audio bytes or iterable of bytes * `filename` - Path to save the audio file **Examples**: ```python theme={null} from fishaudio import FishAudio, save client = FishAudio(api_key="...") audio = client.tts.convert(text="Hello world") # Save to file save(audio, "output.mp3") # Works with iterators too audio_stream = client.tts.convert(text="Another example") save(audio_stream, "another.mp3") ``` # fishaudio.utils.stream Audio streaming utility. #### stream ```python theme={null} def stream(audio_stream: Iterator[bytes]) -> bytes ``` Stream audio in real-time while playing it with mpv. This function plays the audio as it's being generated and simultaneously captures it to return the complete audio buffer. **Arguments**: * `audio_stream` - Iterator of audio byte chunks **Returns**: Complete audio bytes after streaming finishes **Raises**: * `DependencyError` - If mpv is not installed **Examples**: ```python theme={null} from fishaudio import FishAudio, stream client = FishAudio(api_key="...") audio_stream = client.tts.convert(text="Hello world") # Stream and play in real-time, get complete audio complete_audio = stream(audio_stream) # Save the captured audio with open("output.mp3", "wb") as f: f.write(complete_audio) ``` # Legacy Source: https://docs.fish.audio/archive/python-sdk-legacy/index Archived documentation for the legacy Session-based Python SDK This documentation is for the legacy Python SDK using the Session-based API. This API is deprecated. **Please migrate to the [new Python SDK](/developer-guide/sdk-guide/python)** which uses a modern client-based architecture. See the [migration guide](/archive/python-sdk-legacy/migration-guide) for help upgrading. ## About the Legacy SDK This archive contains documentation for the `fish_audio_sdk` module using the Session-based API. While this API still functions, it is no longer actively maintained and lacks the modern features available in the new SDK. ### What's Different in the New SDK The new Python SDK (`fishaudio` module) offers: * **Modern client-based architecture** - More intuitive and consistent with modern Python libraries * **Full async support** - Native asyncio integration for better performance * **Better type safety** - Comprehensive type hints and better IDE support * **Improved error handling** - More detailed error messages and exception hierarchy * **Enhanced utilities** - Built-in audio playback, streaming, and file management * **Active maintenance** - Regular updates and new features ### Migration Path We strongly recommend migrating to the new SDK. The [migration guide](/archive/python-sdk-legacy/migration-guide) provides: * Side-by-side code comparisons * Complete list of breaking changes * Common migration patterns * Troubleshooting tips ## Migration Complete guide to upgrading from the legacy SDK to the new client-based API ## Legacy Documentation Pages How to install the legacy SDK Session initialization and API keys TTS with the Session-based API Reference audio and voice models ASR transcription with legacy API Real-time streaming with WebSocketSession # Contributing Source: https://docs.fish.audio/contributing Help improve Fish Audio and contribute to our open source projects. # Contributing to Fish Audio First off, thanks for taking the time to contribute! All types of contributions are encouraged and valued. See the sections below for different ways to help and details about how this project handles them. Please make sure to read the relevant section before making your contribution. It will make it a lot easier for us maintainers and smooth out the experience for all involved. The community looks forward to your contributions. If you like the project but don't have time to contribute, there are other easy ways to support Fish Audio: * Star our repositories * Tweet about it * Reference Fish Audio in your project's readme * Mention the project at local meetups and tell your friends/colleagues ## Code of Conduct This project and everyone participating in it is governed by the Fish Audio Code of Conduct. By participating, you are expected to uphold this code. Please report unacceptable behavior to our community team. ## I Have a Question Before you ask a question, please read the available [Documentation](https://docs.fish.audio). It's best to search for existing [Issues](https://github.com/fishaudio) that might help you. In case you have found a suitable issue and still need clarification, you can write your question in that issue. It is also advisable to search the internet for answers first. If you still need to ask a question: 1. Open an [Issue](https://github.com/fishaudio) in the relevant repository 2. Provide as much context as you can about what you're running into 3. Provide project and platform versions (Node.js, Python, OS, etc.), depending on what seems relevant We will take care of the issue as soon as possible. ## I Want To Contribute **Legal Notice** When contributing to this project, you must agree that you have authored 100% of the content, that you have the necessary rights to the content, and that the content you contribute may be provided under the project license. ### Reporting Bugs #### Before Submitting a Bug Report A good bug report shouldn't leave others needing to chase you up for more information. Please investigate carefully, collect information, and describe the issue in detail: * Make sure you are using the latest version * Determine if your bug is really a bug and not an error on your side (e.g., incompatible environment components/versions) * Check if there is already a bug report for your issue in the bug tracker * Search the internet (including Stack Overflow) to see if others have discussed the issue * Collect information about the bug: * Stack trace (Traceback) * OS, Platform and Version (Windows, Linux, macOS, x86, ARM) * Version of the interpreter, compiler, SDK, runtime environment, package manager * Your input and the output * Can you reliably reproduce the issue? Can you reproduce it with older versions? #### How Do I Submit a Good Bug Report? You must never report security-related issues, vulnerabilities, or bugs including sensitive information to the issue tracker. Instead, sensitive bugs must be sent by email to our security team. We use GitHub issues to track bugs and errors. If you run into an issue: 1. Open an [Issue](https://github.com/fishaudio) in the relevant repository 2. Explain the behavior you would expect and the actual behavior 3. Provide as much context as possible and describe the **reproduction steps** that someone else can follow to recreate the issue on their own 4. Provide the information you collected in the previous section Once filed: * The project team will label the issue accordingly * A team member will try to reproduce the issue with your provided steps * If there are no reproduction steps, the team will ask for them and mark the issue as `needs-repro` * If the team reproduces the issue, it will be marked `needs-fix` and left to be implemented ### Suggesting Enhancements This section guides you through submitting an enhancement suggestion for Fish Audio, including completely new features and minor improvements to existing functionality. #### Before Submitting an Enhancement * Make sure you are using the latest version * Read the [documentation](https://docs.fish.audio) carefully to see if the functionality already exists * Perform a [search](https://github.com/fishaudio) to see if the enhancement has already been suggested * Consider whether your idea fits with the scope and aims of the project #### How Do I Submit a Good Enhancement Suggestion? Enhancement suggestions are tracked as GitHub issues: * Use a **clear and descriptive title** for the issue * Provide a **step-by-step description** of the suggested enhancement in as many details as possible * **Describe the current behavior** and **explain which behavior you expected to see instead** and why * Include **screenshots or screen recordings** if applicable * **Explain why this enhancement would be useful** to most Fish Audio users ### Your First Code Contribution We welcome first-time contributors! Here's how to get started: 1. **Fork the repository** you want to contribute to 2. **Clone your fork** locally 3. **Create a new branch** for your changes 4. **Make your changes** following our styleguides 5. **Test your changes** thoroughly 6. **Commit your changes** with clear commit messages 7. **Push to your fork** and submit a pull request Look for issues labeled `good first issue` or `help wanted` for beginner-friendly tasks. ### Improving The Documentation Documentation improvements are always welcome! This includes: * Fixing typos and grammatical errors * Adding missing information or clarifications * Improving code examples * Adding new guides or tutorials * Translating documentation See our [documentation repository](https://github.com/fishaudio/fish-docs) to get started. ## Styleguides ### Commit Messages * Use clear and meaningful commit messages * Start with a verb in the present tense (e.g., "Add", "Fix", "Update", "Remove") * Keep the first line under 72 characters * Reference issues and pull requests when relevant * Provide additional context in the commit body if needed Example: ``` Add voice cloning support for Python SDK - Implement VoiceCloneClient class - Add comprehensive error handling - Include usage examples in docstrings Closes #123 ``` ### Code Style * Follow the existing code style in each repository * Use meaningful variable and function names * Add comments for complex logic * Write tests for new features * Ensure all tests pass before submitting ## Attribution This contribution guide is based on the **contributing.md** generator. Fish Audio is committed to open source and welcomes contributions from developers worldwide. # Emotion & Expression Control Source: https://docs.fish.audio/developer-guide/best-practices/emotion-control Make your AI voices express emotions naturally ## Overview Control how your AI voice expresses emotions, from happy and excited to sad and contemplative. Add natural pauses, laughter, and other human-like elements to make speech more engaging. The `(parenthesis)` syntax on this page applies to the S1 model. S2 uses `[bracket]` syntax with natural language descriptions and is not limited to a fixed set of tags. See the [Models Overview](/developer-guide/models-pricing/models-overview#s2-natural-language-control) for details. ## How to Use Simply wrap emotion tags in parentheses before your text: ``` (happy) What a beautiful day! (sad) I'm sorry to hear that. (excited) This is amazing news! ``` Include tone markers or audio effects: ``` (whispering) Let me tell you something. (laughing) Ha ha ha, wow that's so funny! ``` ## Important Rules ### Placement Matters **For all languages:** * Emotion tags MUST go at the beginning of sentences * Tone controls can go anywhere in the text * Sound effects can go anywhere in the text **Correct:** ``` (happy) What a wonderful day! ``` **Incorrect:** ``` What a (happy) wonderful day! ``` ## Best Practices **Do:** * Use one emotion per sentence * Add sounds after relevant words * Keep tags simple and clear * Test different combinations **Don't:** * Overuse tags in short text * Mix conflicting emotions * Create custom tags * Forget the parentheses ## Available Emotions See the [Emotion Control guide](/developer-guide/core-features/emotions) for the full list of supported emotions. ## Scene Examples **Customer Service:** ``` (friendly) Hello! How can I help you today? (empathetic) I understand your frustration. (confident) I'll resolve this for you right away. ``` **Storytelling:** ``` (mysterious)(whispering) Once upon a midnight dreary... (excited) Suddenly, the door burst open! (scared)(shouting) Run for your lives! ``` **Educational Content:** ``` (enthusiastic) Welcome to today's lesson! (curious) Have you ever wondered why the sky is blue? (proud) Great job! You got it right! ``` ## Real-World Examples ### Virtual Assistant ``` (friendly) Good morning! (helpful) I've prepared your schedule for today. (concerned) You have three urgent emails. (encouraging) Let's tackle them together! ``` ### Audiobook Narration ``` (narrator) Chapter One: The Beginning (mysterious) The old house stood silent in the fog. (scared)(whispering) "Is anyone there?" she asked. (relieved)(sighing) No one answered. Phew. ``` ### Game Character ``` (brave) I'll defeat the dragon! (struggling)(panting) This is... harder than... I thought! (triumphant)(shouting) Victory is mine! (laughing) Ha ha ha! ``` ## Advanced Techniques ### Emotion Transitions Gradually change emotions: ``` (happy) I got the promotion! (uncertain) But... it means moving away. (sad) I'll miss everyone here. ``` ### Background Effects Add atmosphere: ``` The comedy show was amazing (audience laughing) Everyone was having fun (background laughter) The crowd loved it (crowd laughing) ``` ## Troubleshooting ### Emotion Not Working? 1. Check tag placement (beginning of sentence for emotions) 2. Verify spelling exactly matches the list 3. Don't use quotes around tags 4. Include parentheses ### Unnatural Sound? * Add appropriate text after sound tags * Don't overuse in short sentences * Space out emotional changes * Test with different voices ### Tips for Success 1. **Start simple** - Use basic emotions first 2. **Preview often** - Test how it sounds 3. **Be consistent** - Keep character emotions logical 4. **Less is more** - Don't overuse tags ## Get Creative Experiment with combinations to create unique character voices and engaging narratives. The key is finding the right balance between emotional expression and natural speech flow. ## Support Need help with emotions? * **Try it live:** [fish.audio](https://fish.audio) * **Community:** [Discord](https://discord.gg/fish-audio) * **Email:** [support@fish.audio](mailto:support@fish.audio) # Real-time Voice Streaming Source: https://docs.fish.audio/developer-guide/best-practices/real-time-streaming Stream voice generation in real-time for interactive applications ## Overview Real-time streaming lets you generate speech as you type or speak, perfect for chatbots, virtual assistants, and live applications. ## When to Use Streaming **Perfect for:** * Live chat applications * Virtual assistants * Interactive storytelling * Real-time translations * Gaming dialogue **Not ideal for:** * Pre-recorded content * Batch processing ## Getting Started ### Web Playground Try real-time streaming instantly: 1. Visit [fish.audio](https://fish.audio) 2. Enable "Streaming Mode" 3. Start typing and hear voice generation in real-time ### Using the SDK Stream text as it's being written: ```python theme={null} from fishaudio import FishAudio # Initialize client client = FishAudio(api_key="your_api_key") # Stream text word by word def stream_text(): text = "Hello, this is being generated in real time" for word in text.split(): yield word + " " # Generate speech as text streams audio_stream = client.tts.stream_websocket( stream_text(), reference_id="your_voice_model_id", temperature=0.7, # Controls variation top_p=0.7, # Controls diversity latency="balanced" ) with open("output.mp3", "wb") as f: for audio_chunk in audio_stream: f.write(audio_chunk) ``` ```javascript theme={null} import { FishAudioClient, RealtimeEvents } from "fish-audio"; import { writeFile } from "fs/promises"; import path from "path"; const apiKey = "your_api_key"; const referenceId = "your_voice_model_id"; async function* makeTextStream() { const chunks = [ "Hello from Fish Audio! ", "This is a realtime text-to-speech test. ", "We are streaming multiple chunks over WebSocket.", ]; for (const chunk of chunks) { yield chunk; await new Promise((r) => setTimeout(r, 200)); } } async function main() { const client = new FishAudioClient({ apiKey }); // For realtime, set text to "" and stream content via makeTextStream const request = { text: "", reference_id: referenceId, }; const connection = await client.textToSpeech.convertRealtime( request, makeTextStream() ); // Collect audio and write to a file when the stream ends const chunks = []; connection.on(RealtimeEvents.OPEN, () => console.log("WebSocket opened")); connection.on(RealtimeEvents.AUDIO_CHUNK, (audio) => { if (audio instanceof Uint8Array || Buffer.isBuffer(audio)) { chunks.push(Buffer.from(audio)); } }); connection.on(RealtimeEvents.ERROR, (err) => console.error("WebSocket error:", err) ); connection.on(RealtimeEvents.CLOSE, async () => { const outPath = path.resolve(process.cwd(), "out.mp3"); await writeFile(outPath, Buffer.concat(chunks)); console.log("Saved to", outPath); }); } main().catch((err) => { console.error(err); process.exit(1); }); ``` ## Configuration Options ### Speed vs Quality **Latency Modes:** * **Normal:** Best quality, \~500ms latency * **Balanced:** Good quality, \~300ms latency ```python theme={null} # Use latency parameter with stream_websocket audio_stream = client.tts.stream_websocket( text_chunks(), reference_id="model_id", latency="balanced" # For faster response ) ``` ```javascript theme={null} const request = { text: "", reference_id: "model_id", latency: "balanced", // For faster response }; ``` ### Voice Control **Temperature** (0.1 - 1.0): * Lower: More consistent, predictable * Higher: More varied, expressive **Top-p** (0.1 - 1.0): * Lower: More focused * Higher: More diverse ## Real-time Applications ### Chatbot Integration Stream responses as they're generated: ```python theme={null} def chatbot_response(user_input): # Get AI response (streaming) ai_text = get_ai_response(user_input) # Convert to speech in real-time audio_stream = client.tts.stream_websocket(ai_text) for audio_chunk in audio_stream: play_audio(audio_chunk) ``` ```javascript theme={null} async function chatbotResponse(userInput) { // Get AI response (streaming) const aiTextStream = getAiResponse(userInput); // async iterable of strings // Convert to speech in real-time for await (const textChunk of aiTextStream) { for await (const audioChunk of ttsStream(textChunk)) { playAudio(audioChunk); } } } ``` ### Live Translation Translate and speak simultaneously: ```python theme={null} def live_translate(source_audio): # Transcribe source audio text = transcribe(source_audio) # Translate text translated = translate(text, target_language) # Stream translated speech for chunk in stream_text(translated): generate_speech(chunk) ``` ```javascript theme={null} async function liveTranslate(sourceAudio) { // Transcribe source audio const text = await transcribe(sourceAudio); // Translate text const translated = await translate(text, targetLanguage); // Stream translated speech for await (const chunk of streamText(translated)) { generateSpeech(chunk); } } ``` ## Best Practices ### Text Buffering **Do:** * Send complete words with spaces * Use punctuation for natural pauses * Buffer 5-10 words for smoothness **Don't:** * Send individual characters * Forget spaces between words * Send huge chunks at once ### Connection Management 1. **Keep connections alive** for multiple generations 2. **Handle disconnections** gracefully 3. **Implement retry logic** for reliability ### Audio Playback For smooth playback: * Buffer 2-3 audio chunks * Use cross-fading between chunks * Handle network delays gracefully ## Common Use Cases ### Interactive Story ```python theme={null} def interactive_story(): story_parts = [ "Once upon a time,", "in a land far away,", "there lived a brave knight..." ] for part in story_parts: # Generate and play each part stream_speech(part) # Wait for user input user_choice = get_user_input() # Continue based on choice ``` ```javascript theme={null} function interactiveStory() { const storyParts = [ "Once upon a time,", "in a land far away,", "there lived a brave knight...", ]; for (const part of storyParts) { // Generate and play each part streamSpeech(part); // Wait for user input const userChoice = getUserInput(); // Continue based on choice } } ``` ### Virtual Assistant ```python theme={null} def virtual_assistant(): while True: # Listen for wake word if detect_wake_word(): # Start streaming response response = process_command() stream_speech(response) ``` ```javascript theme={null} async function virtualAssistant() { while (true) { // Listen for wake word if (detectWakeWord()) { // Start streaming response const response = processCommand(); streamSpeech(response); } } } ``` ### Live Commentary ```python theme={null} def live_commentary(event_stream): for event in event_stream: # Generate commentary commentary = generate_commentary(event) # Stream immediately stream_speech(commentary) ``` ```javascript theme={null} async function liveCommentary(eventStream) { for await (const event of eventStream) { // Generate commentary const commentary = generateCommentary(event); // Stream immediately streamSpeech(commentary); } } ``` ## Troubleshooting ### Audio Gaps **Problem:** Gaps between audio chunks
**Solution:** * Increase buffer size * Use balanced latency mode * Check network connection ### Delayed Response **Problem:** Long wait before audio starts
**Solution:** * Use balanced latency mode * Send initial text immediately * Reduce chunk size ### Choppy Playback **Problem:** Audio cuts in and out
**Solution:** * Buffer more chunks before playing * Check network stability * Use consistent chunk sizes ## Advanced Features ### Dynamic Voice Switching Change voices mid-stream: ```python theme={null} # Start with one voice def text1(): yield "Hello from voice one." audio1 = client.tts.stream_websocket(text1(), reference_id="voice1") for chunk in audio1: play_audio(chunk) # Switch to another def text2(): yield "And now voice two!" audio2 = client.tts.stream_websocket(text2(), reference_id="voice2") for chunk in audio2: play_audio(chunk) ``` ```javascript theme={null} // Start with one voice const request1 = { reference_id: "voice1" }; streamSpeech("Hello from voice one.", request1); // Switch to another const request2 = { reference_id: "voice2" }; streamSpeech("And now voice two!", request2); ``` ### Emotion Injection Add emotions dynamically: ```python theme={null} def emotional_speech(text, emotion): emotional_text = f"({emotion}) {text}" stream_speech(emotional_text) ``` ```javascript theme={null} function emotionalSpeech(text, emotion) { const emotionalText = `(${emotion}) ${text}`; streamSpeech(emotionalText); } ``` ### Speed Control Adjust speaking speed: ```python theme={null} from fishaudio.types import Prosody # Use speed and volume with stream_websocket audio_stream = client.tts.stream_websocket( text_chunks(), speed=1.5 # 1.5x speed ) # Note: For full prosody control including volume, use TTSConfig ``` ```javascript theme={null} const request = { text: "", prosody: { speed: 1.5, // 1.5x speed volume: 0, // Normal volume }, }; ``` ## Performance Tips 1. **Pre-load voices** for instant start 2. **Use connection pooling** for multiple streams 3. **Monitor latency** and adjust settings 4. **Cache common phrases** for instant playback ## Get Support Need help with streaming? * **Discord Community:** [Join our Discord](https://discord.gg/fish-audio) * **Email Support:** [support@fish.audio](mailto:support@fish.audio) * **Status Page:** [status.fish.audio](https://status.fish.audio) # Voice Cloning Best Practices Source: https://docs.fish.audio/developer-guide/best-practices/voice-cloning Simple tips to get the best voice cloning results with Fish Audio ## Getting Started Voice cloning lets you create a digital version of any voice. Use at least 10 seconds of audio recording for studio-quality results right in the Playground or via the API. ## Recording Your Voice ### Find a Quiet Space **Good places to record:** * A bedroom with curtains and carpet * Inside a parked car * A quiet office or study room * Any room with soft furniture **Avoid recording near:** * Open windows with traffic noise * Running appliances (AC, fans, refrigerators) * Other people talking * TVs or music playing ### Use What You Have **Best options:** * USB microphone or gaming headset * Phone voice recorder app (place it on a stable surface) * Earbuds with microphone (hold them steady) **Quick tip:** Keep the microphone about a hand's width from your mouth and speak normally. ## What to Say **Best approach:** Record 2-3 clips of 15-20 seconds each that form a complete paragraph. Here's a sample script you can read naturally: ``` "Hello, my name is Alex, and I enjoy reading books about technology and science. Yesterday, I walked through the park, observing the beautiful autumn leaves. The weather was quite pleasant, with a gentle breeze and warm sunshine. I often think about how amazing our world is, full of interesting discoveries waiting to be made." ``` ### Recording Tips **Must Have:** * Only one person speaking * Steady volume throughout * Consistent tone and emotion * Small pauses between sentences (about half a second) **Nice to Have:** * No background noise * No room echo * Professional mic (but phone is fine too!) **Avoid:** * Multiple speakers in one recording * Big changes in volume or emotion * Background music or TV * Rushing through without pauses ## Troubleshooting ### Common Problems **Voice sounds robotic?** * Try recording for longer, 30-60 seconds * Speak more naturally and add pauses **Voice doesn't sound like you?** * Make sure you're the only person speaking in the recording * Check that there's no background music or TV **Poor audio quality?** * Find a quieter room to record * Move closer to your microphone * Try using a different recording device ## Important: Getting Permission Only clone voices you have permission to use: * Your own voice * Someone who gave you written permission * Never use voices from the internet without permission * Never use celebrity or public figure voices without permission ## How to Upload Your Recording Visit [fish.audio](https://fish.audio) and log in Find the voice creation button in your dashboard Select your recorded file and give your voice a name It usually takes just a few seconds Type some text and hear your cloned voice speak! ## Making Different Voices Want to create character voices or different styles? Try these: ### Different Emotions Record the same text with different feelings: * Happy and energetic * Calm and relaxed * Serious and professional ### Different Characters Create unique voices for: * Storytelling and audiobooks * Game characters * Educational content * Podcast intros ## Get Help Need assistance? We're here to help: * **Community Forum**: [Join our Discord](https://discord.gg/fish-audio) * **Email Support**: [support@fish.audio](mailto:support@fish.audio) * **Video Tutorials**: Coming soon! # Emotion Control Source: https://docs.fish.audio/developer-guide/core-features/emotions Add natural emotions and expressions to your AI-generated speech Drop text with the markers below into the `text` field and send a real request to hear the emotion. ## Overview Fish Audio models support 64+ emotional expressions and voice styles that can be controlled through text markers in your input. Add natural pauses, laughter, and other human-like elements to make speech more engaging and realistic. This page shows S2 usage with `[bracket]` cues. If you use the legacy S1 model, wrap markers in parentheses instead — see [S1 (legacy) syntax](#s1-legacy-syntax) below for the full list, or the [Models Overview](/developer-guide/models-pricing/models-overview#s2-natural-language-control). ## How It Works Add emotional or stylistic cues in square brackets within your text: ```text theme={null} [happy] What a beautiful day! [sad] I'm sorry to hear that. [excited] This is amazing news! ``` The S2 TTS models will interpret these markers and adjust the voice accordingly. ## Complete Emotion Reference ### Basic Emotions (24 expressions) | Emotion | Tag | Description | Example Context | | ----------- | --------------- | ----------------------- | --------------------------- | | Happy | `[happy]` | Cheerful, upbeat tone | Good news, greetings | | Sad | `[sad]` | Melancholic, downcast | Sympathy, bad news | | Angry | `[angry]` | Frustrated, aggressive | Complaints, warnings | | Excited | `[excited]` | Energetic, enthusiastic | Announcements, celebrations | | Calm | `[calm]` | Peaceful, relaxed | Instructions, meditation | | Nervous | `[nervous]` | Anxious, uncertain | Disclaimers, apologies | | Confident | `[confident]` | Assertive, self-assured | Presentations, sales | | Surprised | `[surprised]` | Shocked, amazed | Reactions, discoveries | | Satisfied | `[satisfied]` | Content, pleased | Confirmations, reviews | | Delighted | `[delighted]` | Very pleased, joyful | Celebrations, compliments | | Scared | `[scared]` | Frightened, fearful | Warnings, horror stories | | Worried | `[worried]` | Concerned, troubled | Concerns, questions | | Upset | `[upset]` | Disturbed, distressed | Complaints, problems | | Frustrated | `[frustrated]` | Annoyed, exasperated | Technical issues, delays | | Depressed | `[depressed]` | Very sad, hopeless | Serious topics | | Empathetic | `[empathetic]` | Understanding, caring | Support, counseling | | Embarrassed | `[embarrassed]` | Ashamed, awkward | Apologies, mistakes | | Disgusted | `[disgusted]` | Repelled, revolted | Negative reviews | | Moved | `[moved]` | Emotionally touched | Heartfelt moments | | Proud | `[proud]` | Accomplished, satisfied | Achievements, praise | | Relaxed | `[relaxed]` | At ease, casual | Casual conversation | | Grateful | `[grateful]` | Thankful, appreciative | Thanks, appreciation | | Curious | `[curious]` | Inquisitive, interested | Questions, exploration | | Sarcastic | `[sarcastic]` | Ironic, mocking | Humor, criticism | ### Advanced Emotions (25 expressions) | Emotion | Tag | Description | Example Context | | ------------- | ----------------- | ------------------------ | ---------------------- | | Disdainful | `[disdainful]` | Contemptuous, scornful | Criticism, rejection | | Unhappy | `[unhappy]` | Discontent, dissatisfied | Complaints, feedback | | Anxious | `[anxious]` | Very worried, uneasy | Urgent matters | | Hysterical | `[hysterical]` | Uncontrollably emotional | Extreme reactions | | Indifferent | `[indifferent]` | Uncaring, neutral | Neutral responses | | Uncertain | `[uncertain]` | Doubtful, unsure | Speculation, questions | | Doubtful | `[doubtful]` | Skeptical, questioning | Disbelief, questioning | | Confused | `[confused]` | Puzzled, perplexed | Clarification requests | | Disappointed | `[disappointed]` | Let down, dissatisfied | Unmet expectations | | Regretful | `[regretful]` | Sorry, remorseful | Apologies, mistakes | | Guilty | `[guilty]` | Culpable, responsible | Confessions, apologies | | Ashamed | `[ashamed]` | Deeply embarrassed | Serious mistakes | | Jealous | `[jealous]` | Envious, resentful | Comparisons | | Envious | `[envious]` | Wanting what others have | Admiration with desire | | Hopeful | `[hopeful]` | Optimistic about future | Future plans | | Optimistic | `[optimistic]` | Positive outlook | Encouragement | | Pessimistic | `[pessimistic]` | Negative outlook | Warnings, doubts | | Nostalgic | `[nostalgic]` | Longing for the past | Memories, stories | | Lonely | `[lonely]` | Isolated, alone | Emotional content | | Bored | `[bored]` | Uninterested, weary | Disinterest | | Contemptuous | `[contemptuous]` | Showing contempt | Strong criticism | | Sympathetic | `[sympathetic]` | Showing sympathy | Condolences | | Compassionate | `[compassionate]` | Showing deep care | Support, help | | Determined | `[determined]` | Resolved, decided | Goals, commitments | | Resigned | `[resigned]` | Accepting defeat | Giving up, acceptance | ## Sound & Delivery Markers These markers aren't emotions — they shape *how* a line is delivered, add natural human sounds, or layer in ambient effects. Combine them with the emotion cues above. ### Tone Markers (6 expressions) Control volume, intensity, and emphasis. Place `[emphasis]` right before the word or phrase you want to stress: ```text theme={null} This is [emphasis] really important. ``` | Tone | Tag | Description | When to Use | | ---------- | ------------------- | -------------------- | -------------------------- | | Hurried | `[in a hurry tone]` | Rushed, urgent | Time-sensitive information | | Shouting | `[shouting]` | Loud, calling out | Getting attention | | Screaming | `[screaming]` | Very loud, panicked | Emergencies, fear | | Whispering | `[whispering]` | Very soft, secretive | Secrets, quiet scenes | | Soft | `[soft tone]` | Gentle, quiet | Comfort, lullabies | | Emphasis | `[emphasis]` | Stress a word/phrase | Highlighting key words | ### Audio Effects (11 expressions) Add natural human sounds: | Effect | Tag | Description | Suggested Text | | ------------- | ----------------- | ---------------------------- | -------------- | | Laughing | `[laughing]` | Full laughter | Ha, ha, ha | | Chuckling | `[chuckling]` | Light laugh | Heh, heh | | Sobbing | `[sobbing]` | Crying heavily | Optional text | | Crying Loudly | `[crying loudly]` | Intense crying | Optional text | | Sighing | `[sighing]` | Exhale of relief/frustration | sigh | | Groaning | `[groaning]` | Sound of frustration | ugh | | Panting | `[panting]` | Out of breath | huff, puff | | Gasping | `[gasping]` | Sharp intake of breath | gasp | | Yawning | `[yawning]` | Tired sound | yawn | | Snoring | `[snoring]` | Sleep sound | zzz | | Clear Throat | `[clear throat]` | Throat-clearing sound | ahem | ### Special Effects Additional markers for atmosphere and context: | Effect | Tag | Description | | ------------------- | ----------------------- | ------------------------ | | Audience Laughter | `[audience laughing]` | Crowd laughing sound | | Background Laughter | `[background laughter]` | Ambient laughter | | Crowd Laughter | `[crowd laughing]` | Large group laughing | | Short Pause | `[break]` | Brief pause in speech | | Long Pause | `[long-break]` | Extended pause in speech | You can also use natural expressions like "Ha,ha,ha" for laughter without tags. ## Usage Guidelines ### Placement Rules **For S2:** * Sentence-level emotion cues usually work best at the beginning of sentences * Tone controls can go anywhere in the text * Sound effects can go anywhere in the text * Bracket cues can use natural language descriptions and are not limited to a fixed set of tags **Correct:** ```text theme={null} [happy] What a wonderful day! What a [warm and happy] wonderful day! ``` ## Advanced Techniques ### Combining Effects You can layer multiple emotions for complex expressions: ```text theme={null} [sad][whispering] I miss you so much. [angry][shouting] Get out of here now! [excited][laughing] We won! Ha ha! ``` ### Emotion Transitions Create natural emotional progressions: ```text theme={null} [happy] I got the promotion! [uncertain] But... it means relocating. [sad] I'll miss everyone here. [hopeful] Though it's a great opportunity. [determined] I'm going to make it work! ``` ### Background Effects Add atmospheric sounds: ```text theme={null} The comedy show was amazing [audience laughing] Everyone was having fun [background laughter] The crowd loved it [crowd laughing] ``` ### Intensity Modifiers Fine-tune emotional intensity with descriptive modifiers: ```text theme={null} [slightly sad] I'm a bit disappointed. [very excited] This is absolutely amazing! [extremely angry] This is unacceptable! ``` ## Language Support All 13 supported languages can use emotion markers. For sentence-level control, cues usually work best at the sentence start in these languages: * **English, Chinese, Japanese, German, French, Spanish, Korean, Arabic, Russian, Dutch, Italian, Polish, Portuguese** ## Best Practices ### Do's * Use one primary emotion per sentence * Test different emotion combinations * Match emotions to context logically * Add appropriate text after sound effects (e.g., "Ha ha" after laughing) * Use natural expressions when possible * Space out emotional changes for realism ### Don'ts * Don't overuse emotion tags in short text * Don't mix conflicting emotions * Don't make bracket descriptions so long that they interrupt readability * Don't forget brackets * Don't place sentence-level emotion cues far from the sentence they control ## Common Use Cases ### Customer Service ```text theme={null} [friendly] Hello! How can I help you today? [empathetic] I understand your frustration. [confident] I'll resolve this for you right away. [grateful] Thank you for your patience! ``` ### Storytelling ```text theme={null} [narrator] Once upon a time... [mysterious][whispering] The old house stood silent. [scared] "Is anyone there?" she called out. [relieved][sighing] No one answered. Phew. ``` ### Educational Content ```text theme={null} [enthusiastic] Welcome to today's lesson! [curious] Have you ever wondered why? [encouraging] That's a great question! [proud] Excellent work! ``` ### Marketing & Sales ```text theme={null} [excited] Introducing our newest product! [confident] You won't find better quality anywhere. [urgent] Limited time offer! [satisfied] Join thousands of happy customers! ``` ## Troubleshooting ### Emotion Not Working? 1. **Check placement** - Put the cue where the emotion or effect should begin 2. **Keep wording clear** - Use concise natural language descriptions 3. **Use the right syntax** - S2 cues use square brackets; S1 cues must use parentheses ### Unnatural Sound? * Space out emotional changes * Use appropriate intensity * Test with different voices * Add context text after sound effects ### Performance Notes * Emotion markers don't count toward token limits * No additional latency for emotion processing * All emotions available on all pricing tiers * Maximum of 3 combined emotions per sentence recommended ## Quick Reference Tables ### Emotion Intensity Scale | Base Emotion | Mild | Moderate | Intense | | ------------ | ------------ | -------- | --------- | | Happy | satisfied | happy | delighted | | Sad | disappointed | sad | depressed | | Angry | frustrated | angry | furious | | Scared | nervous | scared | terrified | | Excited | interested | excited | ecstatic | ### Common Combinations | Scenario | Emotion Combo | Example | | ---------------- | -------------------------- | ------------------------------------- | | Whispered Secret | `[mysterious][whispering]` | "I have something to tell you..." | | Angry Shout | `[angry][shouting]` | "Stop right there!" | | Sad Sigh | `[sad][sighing]` | "I wish things were different. Sigh." | | Excited Laugh | `[excited][laughing]` | "We did it! Ha ha!" | | Nervous Question | `[nervous][uncertain]` | "Are you sure about this?" | ## S1 (legacy) syntax The default **S2-Pro** model uses `[bracket]` cues with free-form natural language. The previous-generation **S1** model uses the same emotion names but requires `(parentheses)` and a fixed tag set: ```text theme={null} (happy) What a beautiful day! (sad)(whispering) I'll miss you so much. ``` | Emotion | Tag | Description | Example Context | | ----------- | --------------- | ----------------------- | --------------------------- | | Happy | `(happy)` | Cheerful, upbeat tone | Good news, greetings | | Sad | `(sad)` | Melancholic, downcast | Sympathy, bad news | | Angry | `(angry)` | Frustrated, aggressive | Complaints, warnings | | Excited | `(excited)` | Energetic, enthusiastic | Announcements, celebrations | | Calm | `(calm)` | Peaceful, relaxed | Instructions, meditation | | Nervous | `(nervous)` | Anxious, uncertain | Disclaimers, apologies | | Confident | `(confident)` | Assertive, self-assured | Presentations, sales | | Surprised | `(surprised)` | Shocked, amazed | Reactions, discoveries | | Satisfied | `(satisfied)` | Content, pleased | Confirmations, reviews | | Delighted | `(delighted)` | Very pleased, joyful | Celebrations, compliments | | Scared | `(scared)` | Frightened, fearful | Warnings, horror stories | | Worried | `(worried)` | Concerned, troubled | Concerns, questions | | Upset | `(upset)` | Disturbed, distressed | Complaints, problems | | Frustrated | `(frustrated)` | Annoyed, exasperated | Technical issues, delays | | Depressed | `(depressed)` | Very sad, hopeless | Serious topics | | Empathetic | `(empathetic)` | Understanding, caring | Support, counseling | | Embarrassed | `(embarrassed)` | Ashamed, awkward | Apologies, mistakes | | Disgusted | `(disgusted)` | Repelled, revolted | Negative reviews | | Moved | `(moved)` | Emotionally touched | Heartfelt moments | | Proud | `(proud)` | Accomplished, satisfied | Achievements, praise | | Relaxed | `(relaxed)` | At ease, casual | Casual conversation | | Grateful | `(grateful)` | Thankful, appreciative | Thanks, appreciation | | Curious | `(curious)` | Inquisitive, interested | Questions, exploration | | Sarcastic | `(sarcastic)` | Ironic, mocking | Humor, criticism | | Emotion | Tag | Description | Example Context | | ------------- | ----------------- | ------------------------ | ---------------------- | | Disdainful | `(disdainful)` | Contemptuous, scornful | Criticism, rejection | | Unhappy | `(unhappy)` | Discontent, dissatisfied | Complaints, feedback | | Anxious | `(anxious)` | Very worried, uneasy | Urgent matters | | Hysterical | `(hysterical)` | Uncontrollably emotional | Extreme reactions | | Indifferent | `(indifferent)` | Uncaring, neutral | Neutral responses | | Uncertain | `(uncertain)` | Doubtful, unsure | Speculation, questions | | Doubtful | `(doubtful)` | Skeptical, questioning | Disbelief, questioning | | Confused | `(confused)` | Puzzled, perplexed | Clarification requests | | Disappointed | `(disappointed)` | Let down, dissatisfied | Unmet expectations | | Regretful | `(regretful)` | Sorry, remorseful | Apologies, mistakes | | Guilty | `(guilty)` | Culpable, responsible | Confessions, apologies | | Ashamed | `(ashamed)` | Deeply embarrassed | Serious mistakes | | Jealous | `(jealous)` | Envious, resentful | Comparisons | | Envious | `(envious)` | Wanting what others have | Admiration with desire | | Hopeful | `(hopeful)` | Optimistic about future | Future plans | | Optimistic | `(optimistic)` | Positive outlook | Encouragement | | Pessimistic | `(pessimistic)` | Negative outlook | Warnings, doubts | | Nostalgic | `(nostalgic)` | Longing for the past | Memories, stories | | Lonely | `(lonely)` | Isolated, alone | Emotional content | | Bored | `(bored)` | Uninterested, weary | Disinterest | | Contemptuous | `(contemptuous)` | Showing contempt | Strong criticism | | Sympathetic | `(sympathetic)` | Showing sympathy | Condolences | | Compassionate | `(compassionate)` | Showing deep care | Support, help | | Determined | `(determined)` | Resolved, decided | Goals, commitments | | Resigned | `(resigned)` | Accepting defeat | Giving up, acceptance | | Tone | Tag | Description | When to Use | | ---------- | ------------------- | -------------------- | -------------------------- | | Hurried | `(in a hurry tone)` | Rushed, urgent | Time-sensitive information | | Shouting | `(shouting)` | Loud, calling out | Getting attention | | Screaming | `(screaming)` | Very loud, panicked | Emergencies, fear | | Whispering | `(whispering)` | Very soft, secretive | Secrets, quiet scenes | | Soft | `(soft tone)` | Gentle, quiet | Comfort, lullabies | | Effect | Tag | Description | Suggested Text | | ------------- | ----------------- | ---------------------------- | -------------- | | Laughing | `(laughing)` | Full laughter | Ha, ha, ha | | Chuckling | `(chuckling)` | Light laugh | Heh, heh | | Sobbing | `(sobbing)` | Crying heavily | (optional) | | Crying Loudly | `(crying loudly)` | Intense crying | (optional) | | Sighing | `(sighing)` | Exhale of relief/frustration | sigh | | Groaning | `(groaning)` | Sound of frustration | ugh | | Panting | `(panting)` | Out of breath | huff, puff | | Gasping | `(gasping)` | Sharp intake of breath | gasp | | Yawning | `(yawning)` | Tired sound | yawn | | Snoring | `(snoring)` | Sleep sound | zzz | | Effect | Tag | Description | | ------------------- | ----------------------- | ------------------------ | | Audience Laughter | `(audience laughing)` | Crowd laughing sound | | Background Laughter | `(background laughter)` | Ambient laughter | | Crowd Laughter | `(crowd laughing)` | Large group laughing | | Short Pause | `(break)` | Brief pause in speech | | Long Pause | `(long-break)` | Extended pause in speech | ## See Also * [API Reference](/api-reference/introduction) - Implementation details * [Text-to-Speech Guide and Best Practices](/features/text-to-speech) # Fine-grained Control Source: https://docs.fish.audio/developer-guide/core-features/fine-grained-control Advanced control over speech generation Put your phoneme or paralanguage tags into the `text` field and send a real request to hear the result. ## Getting Started To use fine-grained control, you can use either our SDK, API, or Playground. SDK/API: Phoneme tags are preserved by text normalization, so you can keep the default normalization behavior for pronunciation control. Set `"normalize": false` only when you want to prevent normalization from rewriting the surrounding text, such as numbers, dates, or URLs. Playground: You can use V1.6 Control Model, without setting any other options. Disabling normalization may reduce the stability of reading numbers, dates, and URLs. You'll need to handle these cases manually for best results. ## Phoneme Control Phoneme control allows you to specify exact pronunciations for words, characters, or short phrases. Wrap the desired pronunciation in `<|phoneme_start|>` and `<|phoneme_end|>` tags. The replacement scope depends on the language: * English: replace one word with CMU Arpabet. * Chinese: replace one character or syllable with tone-number pinyin. * Japanese: replace a short Japanese word or phrase with OpenJTalk-style romaji and pitch accent markers. CMU Arpabet examples for names, homographs, acronyms, and technical terms. Tone-number pinyin examples for multi-character words, tones, and polyphonic characters. OpenJTalk romaji phonemes with pitch accent digits. ### Quick Examples English: ```text theme={null} I am an <|phoneme_start|>EH1 N JH AH0 N IH1 R<|phoneme_end|>. ``` Chinese: ```text theme={null} 我是一个<|phoneme_start|>gong1<|phoneme_end|><|phoneme_start|>cheng2<|phoneme_end|><|phoneme_start|>shi1<|phoneme_end|>。 ``` Japanese: ```text theme={null} <|phoneme_start|>ha0shi1ga0<|phoneme_end|>見えます。 ``` ## Paralanguage Paralanguage controls allow you to add natural speech elements and pauses to make the generated speech sound more human-like. There are two main types of controls: ### Pause Words You can use common pause words like "um", "uh", "嗯", "啊" to control the rhythm of the speech. ### Special Effects The following special effects can be added using parentheses: | Effect | Description | First Available | Stage | | ---------------- | ------------------ | --------------- | ------------ | | `(break)` | Short pause | V1.6 | Experimental | | `(long-break)` | Extended pause | V1.6 | Experimental | | `(breath)` | Breathing sound | V1.6 | Experimental | | `(laugh)` | Laughter sound | V1.6 | Experimental | | `(cough)` | Coughing sound | V1.6 | Experimental | | `(lip-smacking)` | Lip smacking sound | V1.6 | Experimental | | `(sigh)` | Sighing sound | V1.6 | Experimental | The effects `(laugh)`, `(cough)`, `(lip-smacking)`, and `(sigh)` are developing. You may need to repeat them multiple times for better results. Example: ```text theme={null} I am, um, an (break) engineer. ``` You can combine paralanguage and phoneme control in the same text: ```text theme={null} I am, um, an (break) <|phoneme_start|>EH1 N JH AH0 N IH1 R<|phoneme_end|>. ``` # Chinese Phoneme Control Source: https://docs.fish.audio/developer-guide/core-features/fine-grained-control/chinese Control Chinese pronunciation with tone-number pinyin ## Overview Chinese phoneme control uses pinyin with tone numbers, also known as tone3 pinyin. Wrap one syllable in each `<|phoneme_start|>` and `<|phoneme_end|>` tag. ```text theme={null} 我是一个<|phoneme_start|>gong1<|phoneme_end|><|phoneme_start|>cheng2<|phoneme_end|><|phoneme_start|>shi1<|phoneme_end|>。 ``` This format is especially useful for polyphonic characters, names, and domain-specific terms where the default reading may be ambiguous. ## Tone Numbers Put the tone number at the end of each pinyin syllable: | Tone | Example | Description | | ---- | ------- | ----------- | | 1 | `ma1` | High level | | 2 | `ma2` | Rising | | 3 | `ma3` | Dipping | | 4 | `ma4` | Falling | | 5 | `ma5` | Neutral | Use lowercase pinyin and keep punctuation outside the phoneme tag. ## Multi-character Words For a multi-character word, place adjacent phoneme tags in the same order as the original characters: ```text theme={null} Standard: 我是一个工程师。 With phoneme control: 我是一个<|phoneme_start|>gong1<|phoneme_end|><|phoneme_start|>cheng2<|phoneme_end|><|phoneme_start|>shi1<|phoneme_end|>。 ``` You can also tag only the ambiguous character and leave the rest of the sentence unchanged: ```text theme={null} 请把这个字读作<|phoneme_start|>hang2<|phoneme_end|>。 ``` ## Polyphonic Characters For polyphonic characters, choose the pinyin that matches the phrase meaning: ```text theme={null} 重庆: <|phoneme_start|>chong2<|phoneme_end|><|phoneme_start|>qing4<|phoneme_end|> 重要: <|phoneme_start|>zhong4<|phoneme_end|><|phoneme_start|>yao4<|phoneme_end|> ``` ```text theme={null} 银行: <|phoneme_start|>yin2<|phoneme_end|><|phoneme_start|>hang2<|phoneme_end|> 行走: <|phoneme_start|>xing2<|phoneme_end|><|phoneme_start|>zou3<|phoneme_end|> ``` ```text theme={null} 音乐: <|phoneme_start|>yin1<|phoneme_end|><|phoneme_start|>yue4<|phoneme_end|> 快乐: <|phoneme_start|>kuai4<|phoneme_end|><|phoneme_start|>le4<|phoneme_end|> ``` ## Generate Pinyin The training pipeline uses the `pypinyin` dictionary and converts entries to tone3 pinyin. The helper below mirrors that behavior for single characters: ```bash theme={null} pip install pypinyin ``` ```python theme={null} from pypinyin.contrib.tone_convert import to_tone3 from pypinyin.pinyin_dict import pinyin_dict def chinese_char_to_pinyin(char: str) -> str | None: pinyin = pinyin_dict.get(ord(char)) if pinyin is None: return None if "," in pinyin: raise ValueError(f"{char} has multiple readings; choose one manually") return to_tone3(pinyin) print(chinese_char_to_pinyin("工")) # gong1 ``` Phrase-level words can require a phrase dictionary or manual selection. For example, `重` should be `chong2` in `重庆` but `zhong4` in `重要`. ## Practical Tips * Use one phoneme tag per Chinese character or syllable. * Keep Chinese punctuation, brackets, and spaces outside the tag. * Choose readings manually for names and polyphonic characters. * Use `ma5`-style tone 5 when you need to mark a neutral tone explicitly. # English Phoneme Control Source: https://docs.fish.audio/developer-guide/core-features/fine-grained-control/english Control English pronunciation with CMU Arpabet ## Overview English phoneme control uses CMU Arpabet, the pronunciation format used by CMUdict. Wrap the pronunciation for one word in `<|phoneme_start|>` and `<|phoneme_end|>`, and keep surrounding punctuation outside the tag. ```text theme={null} I am an <|phoneme_start|>EH1 N JH AH0 N IH1 R<|phoneme_end|>. ``` IPA is not supported for English phoneme tags. Convert IPA pronunciations to CMU Arpabet before using phoneme control. ## CMU Arpabet CMU Arpabet is written as space-separated uppercase symbols. Vowels can include stress digits: * `0` for unstressed vowels. * `1` for primary stress. * `2` for secondary stress. For the full symbol inventory, see the CMUdict [`cmudict.symbols`](https://github.com/cmusphinx/cmudict/blob/master/cmudict.symbols) list. You can also look up words on the [CMU Pronouncing Dictionary](http://www.speech.cs.cmu.edu/cgi-bin/cmudict) page. Example: ```text theme={null} Standard: I am an engineer. With phoneme control: I am an <|phoneme_start|>EH1 N JH AH0 N IH1 R<|phoneme_end|>. ``` You can omit stress digits when you only need a rough pronunciation, but CMUdict-style output with stress digits usually gives the model the clearest signal. ## Common Examples Use phoneme control when spelling alone is ambiguous: ```text theme={null} The <|phoneme_start|>R IY1 D<|phoneme_end|> endpoint returns the current state. The book was <|phoneme_start|>R EH1 D<|phoneme_end|> yesterday. ``` ```text theme={null} The <|phoneme_start|>B EY1 S<|phoneme_end|> line is too loud. The <|phoneme_start|>B AE1 S<|phoneme_end|> swam upstream. ``` ```text theme={null} The <|phoneme_start|>P OW1 L IH0 SH<|phoneme_end|> team joined the call. Please <|phoneme_start|>P AA1 L IH0 SH<|phoneme_end|> the final mix. ``` Use it for product names, acronyms, and technical terms: ```text theme={null} Deploy with <|phoneme_start|>K UW2 B ER0 N EH1 T IY0 Z<|phoneme_end|>. The query uses <|phoneme_start|>EH1 S K Y UW1 EH1 L<|phoneme_end|>. ``` ## Generate CMU Arpabet The training pipeline uses CMUdict-style pronunciations. You can generate the same format with the `cmudict` package: ```bash theme={null} pip install cmudict ``` ```python theme={null} import cmudict entries = cmudict.dict() def cmu_pronunciation(word: str) -> str | None: phones = entries.get(word.lower()) if not phones: return None return " ".join(phones[0]) print(cmu_pronunciation("engineer")) # EH1 N JH AH0 N IH1 R ``` CMUdict may contain multiple pronunciations for the same word. Listen to the result and choose the variant that matches your intended accent or context. ## Practical Tips * Replace only the word whose pronunciation needs control. * Strip punctuation before dictionary lookup, then place punctuation after the tag. * Use CMU Arpabet for English phoneme tags. * For names and brands, write the pronunciation that you want the listener to hear, not necessarily the spelling. # Japanese Phoneme Control Source: https://docs.fish.audio/developer-guide/core-features/fine-grained-control/japanese Control Japanese pronunciation with romaji phonemes and pitch accent markers ## Overview Japanese phoneme control uses OpenJTalk-style romaji phonemes plus pitch accent information. This is useful for Japanese homographs that have the same plain phoneme sequence but different pitch accents, such as `端が`, `箸が`, and `橋が`. ```text theme={null} Standard: 橋が見えます。 With phoneme control: <|phoneme_start|>ha0shi1ga0<|phoneme_end|>見えます。 ``` Unlike Chinese, Japanese phoneme control is usually applied to a short word or phrase, not one tag per character. ## Format Put the pitch level digit immediately after each vowel-bearing mora: * `0` means the current mora is low. * `1` means the current mora is high. * `N` can also carry a pitch digit. * Consonants are written without spaces before the vowel they belong to, for example `ha`, `shi`, and `ga`. * Use OpenJTalk phoneme symbols such as `a`, `i`, `u`, `e`, `o`, `N`, `cl`, `ky`, `sh`, `ch`, and `ts`. The following examples all share the plain phoneme sequence `h a sh i g a`, but the pitch markers disambiguate the word: * `端が` (end + subject marker): `<|phoneme_start|>ha0shi1ga1<|phoneme_end|>` * `箸が` (chopsticks + subject marker): `<|phoneme_start|>ha1shi0ga0<|phoneme_end|>` * `橋が` (bridge + subject marker): `<|phoneme_start|>ha0shi1ga0<|phoneme_end|>` Japanese pitch accent depends on the dictionary, reading, and dialect. Generate the phoneme string from the same text you send to TTS, then listen and adjust the digits when you need a specific accent. ## Relation to ttslearn Prosody Symbols The [ttslearn Japanese Tacotron recipe](https://r9y9.github.io/ttslearn/latest/notebooks/ch10_Recipe-Tacotron.html#%E3%83%95%E3%83%AB%E3%82%B3%E3%83%B3%E3%83%86%E3%82%AD%E3%82%B9%E3%83%88%E3%83%A9%E3%83%99%E3%83%AB%E3%81%8B%E3%82%89%E3%81%AE%E9%9F%B3%E7%B4%A0%E5%88%97%E3%81%8A%E3%82%88%E3%81%B3%E9%9F%BB%E5%BE%8B%E8%A8%98%E5%8F%B7%E3%81%AE%E6%8A%BD%E5%87%BA) shows how to extract phonemes and prosody symbols from OpenJTalk full-context labels. That recipe prints symbols such as `[` for a pitch rise and `]` for a pitch fall. Fish Audio phoneme tags should not contain literal `[` or `]`. Convert that prosody into digit notation, such as `ha0shi1ga0`. ## Generate Japanese Phonemes You can generate Japanese phoneme strings with `pyopenjtalk`. The converter below follows the same full-context label logic used in training: ```bash theme={null} pip install pyopenjtalk ``` ```python theme={null} import re import pyopenjtalk JAPANESE_VOWELS = "aiueoAIUEON" def japanese_to_romaji_with_accent(sentence: str) -> str: text = "" labels = pyopenjtalk.extract_fullcontext(sentence) level = -1 for index, label in enumerate(labels): phoneme = re.search(r"\-([^\+]*)\+", label).group(1) if phoneme in ["sil", "pau"]: continue text += phoneme a1 = int(re.search(r"/A:(\-?[0-9]+)\+", label).group(1)) a2 = int(re.search(r"\+(\d+)\+", label).group(1)) a3 = int(re.search(r"\+(\d+)/", label).group(1)) next_phoneme = re.search(r"\-([^\+]*)\+", labels[index + 1]).group(1) if next_phoneme in ["sil", "pau"]: a2_next = -1 else: a2_next = int(re.search(r"\+(\d+)\+", labels[index + 1]).group(1)) # Accent phrase boundary if a3 == 1 and a2_next == 1: if level >= 0: text += str(level) level = -1 # Falling elif a1 == 0 and a2_next == a2 + 1: level = 0 text += "1" # Rising elif a2 == 1 and a2_next == 2: level = 1 text += "0" elif phoneme in JAPANESE_VOWELS: if level < 0: level = 0 text += str(level) return text print(japanese_to_romaji_with_accent("橋が")) # ha0shi1ga0 ``` Then place the result inside the phoneme tags: ```text theme={null} <|phoneme_start|>ha0shi1ga0<|phoneme_end|> ``` Minimal request body: ```json theme={null} { "text": "<|phoneme_start|>ha0shi1ga0<|phoneme_end|>見えます。" } ``` ## Processing Longer Text For long Japanese text, split on punctuation and tag short Japanese runs instead of wrapping an entire paragraph. The training augmentation used short segments and skipped empty or very long spans. Good: ```text theme={null} <|phoneme_start|>ha0shi1ga0<|phoneme_end|>、見えます。 ``` Avoid: ```text theme={null} <|phoneme_start|>very long paragraph with multiple clauses...<|phoneme_end|> ``` If your text contains symbols that OpenJTalk should read as words, normalize them before conversion. For example, the training preprocessor converted `％` to `パーセント` before extracting phonemes. # Get Your API Key Source: https://docs.fish.audio/developer-guide/getting-started/api-key Create a Fish Audio account, generate an API key, and make your first request Everything you build with Fish Audio — the API, the Python library, JavaScript — authenticates with a single **API key**. Here's how to get one and make your first call in a couple of minutes. ## 1. Create an account and key Go to [fish.audio/auth/signup](https://fish.audio/auth/signup), create an account, and verify your email. Sign in and open [fish.audio/app/api-keys](https://fish.audio/app/api-keys). Click **Create New Key**, give it a descriptive name (and an expiration if you want), then **copy the key and store it securely** — treat it like a password. Never commit your API key to version control or share it publicly. ## 2. Store it as an environment variable The SDKs and the examples throughout these docs read your key from `FISH_API_KEY`: ```bash theme={null} export FISH_API_KEY="your_api_key_here" ``` This keeps the key out of your code and lets you use different keys for development and production. ## 3. Make your first request ```python Python theme={null} from fishaudio import FishAudio from fishaudio.utils import save client = FishAudio() # reads FISH_API_KEY audio = client.tts.convert(text="Hello from Fish Audio!") save(audio, "hello.mp3") ``` ```bash API (curl) theme={null} curl --request POST https://api.fish.audio/v1/tts \ --header "Authorization: Bearer $FISH_API_KEY" \ --header "Content-Type: application/json" \ --header "model: s2-pro" \ --data '{ "text": "Hello from Fish Audio!", "format": "mp3" }' \ --output hello.mp3 ``` You just generated your first audio. Where to next: Voices, formats, speed, and streaming. A fuller first-request walkthrough. Build a custom voice from your own audio. Transcribe audio with timestamps. **Building with an AI coding agent?** Install the Fish Audio skill so it writes correct SDK/API code — `npx skills add docs.fish.audio`. See [AI Coding Agents](/developer-guide/resources/coding-agents). # Changelog Source: https://docs.fish.audio/developer-guide/getting-started/changelog Complete release history and version updates for all Fish Audio products ## Fish Audio S2 Next-generation text-to-speech model with inline emotion cues, multi-speaker dialogue support, and 80+ languages. S2 introduces `[bracket]` syntax for natural language control over emotion and paralinguistic cues (e.g., `[whisper]`, `[laugh]`, `[emphasis]`). Tags are treated as standard text rather than dedicated control tokens, so you are not limited to a fixed set of expressions. Built on the Qwen3-4B backbone and fully open-source. Use model ID `s2-pro` in the API. S1 remains supported for existing integrations. [GitHub](https://github.com/fishaudio/fish-speech) | [HuggingFace](https://huggingface.co/fishaudio) ## Fish Audio S1 Historic rebrand from Fish Speech to Fish Audio. #1 ranking on TTS-Arena2 with industry-leading performance. S1 (4B params): 0.008 WER, 0.004 CER - Available on Fish Audio Playground S1-mini (0.5B params): 0.011 WER, 0.005 CER - Open source on Hugging Face 64+ emotional expressions with RLHF integration and multilingual support for English, Chinese, Japanese, and more. [Read More about S1](https://fish.audio/blog/introducing-s1/) ## v1.5.1 Fixed critical PyTorch security settings and improved inference speed significantly. Added ONNX export support for better deployment options and enhanced text processing for Arabic and Hebrew languages. Includes bug fixes for Apple Silicon (MPS) compatibility and reorganized library structure for cleaner codebase. ## v1.5.0 Introduced v1.5 model architecture with improved dataset handling and bearer token authentication for APIs. Added reference audio caching by hash for faster performance and better Apple Silicon support. Includes OpenAPI documentation refactoring and base64 reference data support in JSON format. ## v1.4.3 Introduced Fish Agent for conversational AI with streaming capabilities and real-time interactions. Added comprehensive Korean language documentation and fixed critical non-English speech issues. Improved WebUI streaming functionality and PyTorch version compatibility. ## v1.4.2 Documentation-focused release with comprehensive updates for v1.4, macOS support, and multiple language translations. Improved Docker support and API enhancements for JSON format handling. Added audio selection to WebUI and fixed various stability issues including cache handling and backend performance. ## v1.4.1 Infrastructure improvements focused on Docker optimization and multi-platform builds. Updated PyTorch version and replaced audio backend from sox for better performance. Enhanced CI/CD pipeline with buildx support and fixed various Docker-related issues. ## v1.4.0 Major release with new VQGAN architecture for improved audio quality and faster inference. Updated WebUI with enhanced interface and better language switching. Added Japanese documentation translation and fixed inference warmup issues for better performance. ## v1.2.1 Replaced Whisper with SenseVoice for better ASR and added native Apple Silicon support. Includes Portuguese (Brazil) localization, streaming audio functionality, and CPU-only inference improvements. Pinned PyTorch to 2.3.1 to fix inference speed issues and aligned API with official closed-source version. ## v1.2 Introduced auto-reranking system for better results along with bilingual support and model quantization. Replaced standard Whisper with Faster Whisper for improved speed and added Japanese documentation. Enhanced model stability and inference performance with optimized v1.2 architecture. ## v1.1.2 Minor release adding Chinese text normalization support and a streaming audio download button in the WebUI. Fixed LoRA merging issues and improved Firefly performance. ## v1.1.1 Breaking changes: Replaced zibai with uvicorn for API server, new text-splitter with byte-based length calculation, and license change to CC-BY-NC-SA 4.0. Added Apple Silicon (MPS) support, Windows one-click installation, and automatic model downloading with resume capability. Improved WebUI with better file selection and download progress indicators. ## v1.1.0 Added VITS decoder integration with full streaming support and queue management for real-time audio generation. Introduced internationalization (i18n) with Spanish translation and improved Windows packaging. Optimized GPU memory usage and CPU-only inference performance while adding LoRA support to the Gradio UI. ## v1.0.0 Major milestone release introducing new VQ-GAN architecture with VITS decoder support, LoRA fine-tuning, and streaming inference capabilities. Breaking changes include removal of the Rust-based data server, new tokenizer replacing phonemizer, and updated model architecture (VQ + DiT + Reflow). Achieved 4x memory reduction during loading and added WebUI for training and annotation. ## v0.2.0 First public release of Fish Speech featuring a complete text-to-speech pipeline with VQ-GAN audio codec and LLAMA-based language model. Includes multi-language support (Chinese, English, Japanese), Gradio WebUI for inference, HTTP API server, and Docker support. Added special optimizations for Chinese users including mirror downloads and localized documentation. # Quick Start Source: https://docs.fish.audio/developer-guide/getting-started/quickstart Generate your first AI voice with Fish Audio in under 5 minutes ## Overview This guide will walk you through generating your first text-to-speech audio with Fish Audio. By the end, you'll have converted text into natural-sounding speech using our API. ## Prerequisites Sign up for a free Fish Audio account to get started with our API. 1. Go to [fish.audio/auth/signup](https://fish.audio/auth/signup) 2. Fill in your details to create an account, complete steps to verify your account. 3. Log in to your account and navigate to the [API section](https://fish.audio/app/api-keys) Once you have an account, you'll need an API key to authenticate your requests. 1. Log in to your [Fish Audio Dashboard](https://fish.audio/app/api-keys/) 2. Navigate to the API Keys section 3. Click "Create New Key" and give it a descriptive name, set a expiration if desired 4. Copy your key and store it securely Keep your API key secret! Never commit it to version control or share it publicly. ## Your First TTS Request Choose your preferred method to generate speech: Store your API key as an environment variable (recommended approach): ```bash theme={null} export FISH_API_KEY="replace_me" ``` Run this [cURL](https://curl.se/) command to generate your first speech: ```bash theme={null} curl -X POST https://api.fish.audio/v1/tts \ -H "Authorization: Bearer $FISH_API_KEY" \ -H "Content-Type: application/json" \ -H "model: s2-pro" \ -d '{ "text": "Hello! Welcome to Fish Audio. This is my first AI-generated voice.", "format": "mp3" }' \ --output welcome.mp3 ``` The audio has been saved as `welcome.mp3`. You can play it by: * Double-clicking the file or opening it in any media player * Or using the command line: ```bash theme={null} # On macOS afplay welcome.mp3 # On Linux mpg123 welcome.mp3 # On Windows start welcome.mp3 ``` ```bash theme={null} pip install fish-audio-sdk ``` Create a Python script: ```python theme={null} from fishaudio import FishAudio from fishaudio.utils import save # Initialize with your API key client = FishAudio() # reads FISH_API_KEY # Generate speech audio = client.tts.convert(text="Hello! Welcome to Fish Audio.") save(audio, "welcome.mp3") print("✓ Audio saved to welcome.mp3") ``` ```bash theme={null} python generate_speech.py ``` The audio has been saved as `welcome.mp3`. You can play it by: * Double-clicking the file or opening it in any media player * Or using the command line: ```bash theme={null} # On macOS afplay welcome.mp3 # On Linux mpg123 welcome.mp3 # On Windows start welcome.mp3 ``` ```bash theme={null} npm install fish-audio ``` Create a JavaScript script: ```javascript theme={null} import { FishAudioClient } from "fish-audio"; import { writeFile } from "fs/promises"; const fishAudio = new FishAudioClient({ apiKey: process.env.FISH_API_KEY }); const audio = await fishAudio.textToSpeech.convert({ text: "Hello, world!", }); const buffer = Buffer.from(await new Response(audio).arrayBuffer()); await writeFile("welcome.mp3", buffer); console.log("✓ Audio saved to welcome.mp3"); ``` ```bash theme={null} node generate_speech.mjs ``` The audio has been saved as `welcome.mp3`. You can play it by: * Double-clicking the file or opening it in any media player * Or using the command line: ```bash theme={null} # On macOS afplay welcome.mp3 # On Linux mpg123 welcome.mp3 # On Windows start welcome.mp3 ``` ## If your first request fails | Status | Likely cause | Fix | | ------ | ------------------------------- | --------------------------------------------------------------------------------- | | `401` | Invalid or missing API key | Check `FISH_API_KEY` and your key on [API Keys](https://fish.audio/app/api-keys). | | `402` | Out of credits | Top up on [Billing](https://fish.audio/app/billing). | | `400` | Bad `reference_id` / parameters | Verify the voice id; read the error `message`. | | `429` | Rate limit | Wait and retry with backoff. | See [Errors](/api-reference/errors) for the full table and retry handling. ## Customizing Your Voice The examples above use the default voice. To use a different voice, add the `reference_id` parameter with a model ID from [fish.audio](https://fish.audio). You can find the model ID in the URL or use the copy button when viewing any voice. Choose a voice to try: From: [https://fish.audio/m/ca3007f96ae7499ab87d27ea3599956a](https://fish.audio/m/ca3007f96ae7499ab87d27ea3599956a) ```bash theme={null} export REFERENCE_ID="ca3007f96ae7499ab87d27ea3599956a" ``` From: [https://fish.audio/m/802e3bc2b27e49c2995d23ef70e6ac89](https://fish.audio/m/802e3bc2b27e49c2995d23ef70e6ac89) ```bash theme={null} export REFERENCE_ID="802e3bc2b27e49c2995d23ef70e6ac89" ``` Then generate speech with your chosen voice: ```bash theme={null} curl -X POST https://api.fish.audio/v1/tts \ -H "Authorization: Bearer $FISH_API_KEY" \ -H "Content-Type: application/json" \ -H "model: s2-pro" \ -d '{ "text": "This is a custom voice from Fish Audio! You can explore hundreds of different voices on the platform, or even create your own.", "reference_id": "'"$REFERENCE_ID"'", "format": "mp3" }' \ --output custom_voice.mp3 ``` ```python theme={null} import os from fishaudio import FishAudio from fishaudio.utils import save client = FishAudio() # reads FISH_API_KEY # Generate speech with custom voice audio = client.tts.convert( text="This is a custom voice from Fish Audio! You can explore hundreds of different voices on the platform, or even create your own.", reference_id=os.environ.get("REFERENCE_ID") ) save(audio, "custom_voice.mp3") ``` ```javascript theme={null} import { FishAudioClient } from "fish-audio"; import { writeFile } from "fs/promises"; const fishAudio = new FishAudioClient({ apiKey: process.env.FISH_API_KEY }); const audio = await fishAudio.textToSpeech.convert({ text: "This is a custom voice from Fish Audio! You can explore hundreds of different voices on the platform, or even create your own.", reference_id: process.env.REFERENCE_ID, }); const buffer = Buffer.from(await new Response(audio).arrayBuffer()); await writeFile("custom_voice.mp3", buffer); console.log("✓ Audio saved to custom_voice.mp3"); ``` ## Support Need help? Check out these resources: * [API Reference](/api-reference/introduction) - Complete API documentation * [Create a Voice Clone](/api-reference/endpoint/model/create-model) - Create a voice clone model * [Generate Speech](/api-reference/endpoint/openapi-v1/text-to-speech) - Generate realistic speech * [Real-time Streaming](/features/realtime-streaming) - WebSocket for real-time streaming * [Discord Community](https://discord.com/invite/dF9Db2Tt3Y) - Get help from the community * [Support Email](mailto:support@fish.audio) - Contact our support team # LiveKit Source: https://docs.fish.audio/developer-guide/integrations/livekit Build real-time voice AI agents with Fish Audio and LiveKit [LiveKit Agents](https://github.com/livekit/agents) is an open source framework for building real-time voice and multimodal AI agents. It handles streaming audio pipelines, turn detection, interruptions, and LLM orchestration so you can focus on your agent's behavior. Fish Audio integrates with LiveKit through the `fishaudio` plugin, providing text-to-speech synthesis with support for both chunked and real-time WebSocket streaming modes. ## Prerequisites * A [Fish Audio account](https://fish.audio) with an API key * Python 3.9 or higher ## Installation Install LiveKit Agents with Fish Audio support: ```bash theme={null} pip install "livekit-agents[fishaudio]" ``` ## Configuration Set your Fish Audio API key as an environment variable: ```bash theme={null} export FISH_API_KEY=your_api_key_here ``` ## Basic usage Add Fish Audio TTS to your LiveKit agent: ```python theme={null} from livekit.plugins.fishaudio import TTS tts = TTS( reference_id="your_voice_model_id", # Optional: use a specific voice model="s1", sample_rate=24000, latency_mode="balanced" ) ``` ### Key parameters | Parameter | Description | | --------------- | ------------------------------------------------------------------------- | | `api_key` | Your Fish Audio API key (or use `FISH_API_KEY` env var) | | `model` | TTS model/backend to use (default: `s1`) | | `reference_id` | Voice model ID from the [Fish Audio library](https://fish.audio/discover) | | `output_format` | Audio format: `pcm`, `mp3`, `wav`, or `opus` (default: `pcm`) | | `sample_rate` | Audio sample rate in Hz (default: `24000`) | | `num_channels` | Number of audio channels (default: `1`) | | `base_url` | Custom API endpoint (default: `https://api.fish.audio`) | | `latency_mode` | `normal` (\~500ms) or `balanced` (\~300ms, default) | ### Streaming modes The plugin supports two synthesis modes: ```python theme={null} # Chunked (non-streaming) synthesis stream = tts.synthesize("Hello, world!") # Real-time WebSocket streaming stream = tts.stream() ``` ## Resources * [LiveKit Agents Documentation](https://docs.livekit.io/agents/) * [LiveKit GitHub](https://github.com/livekit/agents) * [Fish Audio Plugin Reference](https://docs.livekit.io/reference/python/v1/livekit/plugins/fishaudio/index.html) * [Fish Audio Voice Library](https://fish.audio/discovery) # n8n Source: https://docs.fish.audio/developer-guide/integrations/n8n Automate workflows with Fish Audio and n8n [n8n](https://n8n.io/) is a fair-code licensed workflow automation platform. The Fish Audio community node brings text-to-speech, speech-to-text, and voice cloning capabilities to your n8n workflows. ## Installation Install from n8n community nodes: 1. Go to **Settings** > **Community Nodes** 2. Select **Install** 3. Enter `n8n-nodes-fishaudio` 4. Accept the risks and install See the [n8n community nodes guide](https://docs.n8n.io/integrations/community-nodes/installation/) for details. ## Configuration 1. Go to **Credentials** > **Add Credential** 2. Search for "Fish Audio API" 3. Enter your API key from [fish.audio/app/api-keys](https://fish.audio/app/api-keys) ## Features The node supports: * **Text-to-Speech** — Generate audio from text using any voice model * **Speech-to-Text** — Transcribe audio files * **Voice Models** — List, create, and manage custom voices * **Account** — Check credit balance The node is also available as an AI tool for use with n8n's AI Agent nodes. ## Resources * [npm package](https://www.npmjs.com/package/n8n-nodes-fishaudio) * [GitHub](https://github.com/fishaudio/fish-audio-n8n) * [n8n Community Nodes](https://docs.n8n.io/integrations/community-nodes/) # Pipecat Source: https://docs.fish.audio/developer-guide/integrations/pipecat Build voice AI agents with Fish Audio and Pipecat [Pipecat](https://github.com/pipecat-ai/pipecat) is an open source framework for building voice and multimodal conversational AI. It handles the orchestration of audio, AI services, and conversation pipelines so you can focus on what makes your agent unique. Fish Audio integrates with Pipecat through `FishAudioTTSService`, which provides real-time text-to-speech synthesis using WebSocket streaming for low-latency conversational applications. ## Prerequisites * A [Fish Audio account](https://fish.audio) with an API key * Python 3.9 or higher ## Installation Install Pipecat with Fish Audio support: ```bash theme={null} pip install "pipecat-ai[fish]" ``` ## Configuration Set your Fish Audio API key as an environment variable: ```bash theme={null} export FISH_API_KEY=your_api_key_here ``` ## Basic usage Add `FishAudioTTSService` to your Pipecat pipeline: ```python theme={null} from pipecat.services.fish import FishAudioTTSService tts = FishAudioTTSService( api_key=os.getenv("FISH_API_KEY"), reference_id="your_voice_model_id", # Optional: use a specific voice model_id="s1", params=FishAudioTTSService.InputParams( latency="normal", prosody_speed=1.0 ) ) ``` ### Key parameters | Parameter | Description | | --------------- | ------------------------------------------------------------------------- | | `api_key` | Your Fish Audio API key | | `reference_id` | Voice model ID from the [Fish Audio library](https://fish.audio/discover) | | `model_id` | TTS model version (default: `s1`) | | `output_format` | Audio format: `pcm`, `mp3`, `wav`, or `opus` | ### Prosody controls Customize speech characteristics with `InputParams`: ```python theme={null} params=FishAudioTTSService.InputParams( latency="balanced", # "normal" or "balanced" prosody_speed=1.2, # 0.5 to 2.0 prosody_volume=0, # Volume adjustment in dB normalize=True # Audio normalization ) ``` ## Resources * [Pipecat Documentation](https://docs.pipecat.ai/server/services/tts/fish) * [Pipecat GitHub](https://github.com/pipecat-ai/pipecat) * [Fish Audio Voice Library](https://fish.audio/discovery) # Choosing a Model Source: https://docs.fish.audio/developer-guide/models-pricing/choosing-a-model Select the right Fish Audio model for your use case and requirements We recommend using **Fish Audio S2.1-Pro** for production projects. It improves on S2-Pro quality, latency, and throughput, and is the right choice when you need production TTFA and DPA guarantees. Use **`s2.1-pro-free`** for testing, prototyping, development, and smaller businesses. It is the same model as S2.1-Pro at \$0, but it does not guarantee TTFA or DPA. ## Support Need help? Check out these resources: * [API Reference](/api-reference/introduction) - Complete API documentation * [Create a Voice Clone](/api-reference/endpoint/model/create-model) - Create a voice clone model * [Generate Speech](/api-reference/endpoint/openapi-v1/text-to-speech) - Generate realistic speech * [Real-time Streaming](/features/realtime-streaming) - WebSocket for real-time streaming * [Discord Community](https://discord.com/invite/dF9Db2Tt3Y) - Get help from the community * [Support Email](mailto:support@fish.audio) - Contact our support team # Model Deprecations Source: https://docs.fish.audio/developer-guide/models-pricing/deprecations Track deprecated models and migration timelines for Fish Audio services ## Available Models Currently available models: * **Fish Audio S2** (Recommended) - Latest generation with best performance * **Fish Audio S1** - Highly expressive and natural sounding ## Deprecated Models * **speech-1.6** - Fish Speech v1.6 has been deprecated on February, 28th, 2026 * **speech-1.5** - Fish Speech v1.5 has been deprecated on February, 28th, 2026 We strongly recommend using **Fish Audio S1** for all new projects to access the latest capabilities and performance improvements. ## Support Need help? Check out these resources: * [API Reference](/api-reference/introduction) - Complete API documentation * [Create a Voice Clone](/api-reference/endpoint/model/create-model) - Create a voice clone model * [Generate Speech](/api-reference/endpoint/openapi-v1/text-to-speech) - Generate realistic speech * [Real-time Streaming](/features/realtime-streaming) - WebSocket for real-time streaming * [Discord Community](https://discord.com/invite/dF9Db2Tt3Y) - Get help from the community * [Support Email](mailto:support@fish.audio) - Contact our support team # Models Overview Source: https://docs.fish.audio/developer-guide/models-pricing/models-overview Explore Fish Audio's voice generation models and their capabilities ## Available Models Fish Audio offers state-of-the-art text-to-speech models optimized for different use cases and performance requirements. ### Recommended Model **Fish Audio S2.1-Pro** - Our recommended production TTS model and an improved version of S2-Pro * Natural language control with `[bracket]` syntax — not limited to a fixed set (e.g., `[whispers sweetly]`, `[laughing nervously]`) * Multi-speaker dialogue support * 83 languages * Improved quality, latency, and throughput over S2-Pro * Production option for workloads that need TTFA and DPA guarantees ### Free Development Model **Fish Audio S2.1-Pro Free** - The same model as S2.1-Pro, available at \$0 for development and testing * Use the `s2.1-pro-free` model string with the same TTS API endpoint * Same model quality and language coverage as `s2.1-pro` * Free to use under fair-use limits * No TTFA or DPA guarantees * Best for testing, prototyping, development, and smaller businesses ### Previous S2 Model **Fish Audio S2-Pro** - Previous-generation S2 TTS model * Natural language control with `[bracket]` syntax — not limited to a fixed set (e.g., `[whispers sweetly]`, `[laughing nervously]`) * Multi-speaker dialogue support * 80+ languages * 100ms time-to-first-audio * Full SGLang-based serving stack * Open-source We recommend using `s2.1-pro` for production projects. Use `s2.1-pro-free` when you want the same model for evaluation, prototyping, development, and smaller businesses without TTFA or DPA guarantees. S1 remains available for existing integrations. ### Previous Model **Fish Audio S1** - High-quality voice generation * 4 billion parameters * 0.008 WER (0.8% word error rate) * Full emotional control capabilities with `(parenthesis)` syntax ## Model Specifications ### Fish Audio S1 Performance Metrics * **Word Error Rate (WER)**: 0.008 (0.8%) * **Character Error Rate (CER)**: 0.004 (0.4%) * **Real-time Factor**: \~1:7 on standard hardware * **TTS-Arena2 Ranking**: #1 worldwide ## Supported Languages ### S2.1-Pro and S2-Pro S2.1-Pro supports 83 languages, while S2-Pro supports 80+ languages. Both use automatic language detection and support inline emotion and paralinguistic cues. Language detection is automatic - simply provide text in your target language. ### S1 S1 supports text-to-speech generation in 13 languages with full emotional expression capabilities. ``` English, Chinese, Japanese, German, French, Spanish, Korean, Arabic, Russian, Dutch, Italian, Polish, Portuguese ``` ## Voice Styles and Emotions Fish Audio models support emotional expressions and voice styles that can be controlled through text markers in your input. ### S2.1-Pro and S2-Pro Natural Language Control S2.1-Pro and S2-Pro treat `[bracket]` tags as standard text rather than dedicated control tokens. Through training on massive datasets, the models learned implicit mappings between natural language descriptions and acoustic variations. This means you are not limited to a predefined set of tags — you can use any descriptive expression and the model will interpret it, such as `[whispers sweetly]` or `[laughing nervously]`. Common examples include: ``` [whisper] [laugh] [emphasis] [sigh] [gasp] [pause] [angry] [excited] [sad] [surprised] [inhale] [exhale] ``` S2 cues can be placed anywhere in your text to control emotion at specific positions. For example: `"I can't believe it [gasp] you actually did it [laugh]"` ### S1 Voice Styles and Emotions S1 supports 64+ emotional expressions using `(parenthesis)` syntax. ### Basic Emotions (24 expressions) ``` (angry) (sad) (excited) (surprised) (satisfied) (delighted) (scared) (worried) (upset) (nervous) (frustrated) (depressed) (empathetic) (embarrassed) (disgusted) (moved) (proud) (relaxed) (grateful) (confident) (interested) (curious) (confused) (joyful) ``` ### Advanced Emotions (25 expressions) ``` (disdainful) (unhappy) (anxious) (hysterical) (indifferent) (impatient) (guilty) (scornful) (panicked) (furious) (reluctant) (keen) (disapproving) (negative) (denying) (astonished) (serious) (sarcastic) (conciliative) (comforting) (sincere) (sneering) (hesitating) (yielding) (painful) (awkward) (amused) ``` ### Tone Markers (5 expressions) ``` (in a hurry tone) (shouting) (screaming) (whispering) (soft tone) ``` ### Audio Effects (10 expressions) ``` (laughing) (chuckling) (sobbing) (crying loudly) (sighing) (panting) (groaning) (crowd laughing) (background laughter) (audience laughing) ``` You can also use natural expressions like "Ha,ha,ha" for laughter. Experiment with combinations to achieve the perfect emotional tone for your application. ## Support Need help? Check out these resources: * [API Reference](/api-reference/introduction) - Complete API documentation * [Create a Voice Clone](/api-reference/endpoint/model/create-model) - Create a voice clone model * [Generate Speech](/api-reference/endpoint/openapi-v1/text-to-speech) - Generate realistic speech * [Real-time Streaming](/features/realtime-streaming) - WebSocket for real-time streaming * [Discord Community](https://discord.com/invite/dF9Db2Tt3Y) - Get help from the community * [Support Email](mailto:support@fish.audio) - Contact our support team # Pricing & Rate Limits Source: https://docs.fish.audio/developer-guide/models-pricing/pricing-and-rate-limits Understand Fish Audio pricing plans, usage costs, and API rate limits ## API Pricing The Fish Audio API uses pay-as-you-go pricing based on actual usage. There are no subscription fees or monthly minimums for API access. ### Text-to-Speech (TTS) Models TTS pricing is based on the size of input text, measured in millions of UTF-8 bytes. | Model Name | Price (USD) | | --------------- | ----------------------- | | `s2.1-pro` | \$15.00 / M UTF-8 bytes | | `s2.1-pro-free` | \$0.00 / M UTF-8 bytes | | `s2-pro` | \$15.00 / M UTF-8 bytes | | `s1` | \$15.00 / M UTF-8 bytes | 1M UTF-8 bytes is approximately 180,000 English words, or about 12 hours of speech ### Automatic Speech Recognition (ASR) Models | Model Name | Price (USD) | | -------------- | ------------------- | | `transcribe-1` | \$0.36 / audio hour | **How ASR billing works:** * Charges are based on the duration of audio processed * Duration is rounded up to the nearest second ### Voice Design | Model Name | Price (USD) | | ---------------- | ------------------------------- | | `voice-design-1` | \$0.01 / successful API request | **How Voice Design billing works:** * Charges are based on successful `POST /v1/voice-design` requests * One successful request is charged once, even when it returns multiple candidates * Authentication, validation, balance, concurrency, and service errors are not billed ## Rate Limits These limits help us ensure fair usage and maintain service quality for all users. ### Concurrent Request Limits | Tier | Spending Threshold | Concurrent Requests | | ----------- | ------------------ | ------------------- | | Starter | \< \$100 paid | 5 requests | | Elevated | ≥ \$100 paid | 15 requests | | High Volume | ≥ \$1,000 paid | 50 requests | | Enterprise | Custom | Custom limits | Concurrency tiers unlock as soon as your total prepaid amount reaches the threshold. You do not need to spend the full balance first. If your workload needs a higher concurrency tier, you can top up in advance to unlock the next tier immediately. ### Convert Concurrency to QPS or QPM Fish Audio rate limits are based on **concurrent requests**: the number of requests that can be in progress at the same time. This is different from QPS (queries per second) or QPM (queries per minute), which measure how many requests complete over time. The conversion depends on how long your Fish Audio request occupies a concurrency slot. Short interactive requests can produce much higher QPM from the same concurrency tier than long-form synthesis requests. Use this planning formula: ```text theme={null} QPS ~= concurrency / average_request_duration_seconds QPM ~= concurrency * 60 / average_request_duration_seconds required_concurrency ~= target_QPM * average_request_duration_seconds / 60 ``` Before you estimate throughput, measure your own average request duration: 1. For TTS workloads, send representative production-shaped requests with `latency="normal"` to get a stable quality-first baseline. For other APIs, use the same request shape you expect in production. 2. Measure from request start until the full response completes. For WebSocket streaming, measure until the final audio chunk is received or the stream is closed. 3. Calculate the average duration for each workload type. If one business workflow makes multiple Fish Audio calls, estimate each call separately. Use p95 duration when you need safer capacity planning for bursts, retries, or uneven traffic. There is no single fixed conversion from concurrency to QPM. A customer whose average request occupies a slot for 2 seconds can complete about 30 times as many requests per minute as a customer whose average request occupies a slot for 60 seconds, even on the same concurrency tier. Example planning estimates: | Workload | Average occupied time | 5 concurrency | 15 concurrency | 50 concurrency | | -------------------------------- | --------------------- | ------------- | -------------- | -------------- | | AI companion short reply | 2 seconds | \~150 QPM | \~450 QPM | \~1,500 QPM | | Voice-agent support turn | 4 seconds | \~75 QPM | \~225 QPM | \~750 QPM | | Narration paragraph | 12 seconds | \~25 QPM | \~75 QPM | \~250 QPM | | Audiobook or long-form synthesis | 60 seconds | \~5 QPM | \~15 QPM | \~50 QPM | Treat these numbers as examples, not guarantees. Your actual QPS and QPM depend on text length, model, reference audio, output format, network path, streaming behavior, and the latency distribution of your own application. Please reach out to our team to enable enterprise volume pricing, rate limits, and billing. ## Support Need help? Check out these resources: * [API Reference](/api-reference/introduction) - Complete API documentation * [Create a Voice Clone](/api-reference/endpoint/model/create-model) - Create a voice clone model * [Generate Speech](/api-reference/endpoint/openapi-v1/text-to-speech) - Generate realistic speech * [Real-time Streaming](/features/realtime-streaming) - WebSocket for real-time streaming * [Discord Community](https://discord.com/invite/dF9Db2Tt3Y) - Get help from the community * [Support Email](mailto:support@fish.audio) - Contact our support team # Agent Quickstart Source: https://docs.fish.audio/developer-guide/resources/agent-quickstart Build with Fish Audio using your AI coding agent — install the skill and start prompting in a minute Install the Fish Audio **agent skill** and your coding agent — Claude Code, Cursor, Codex, and others — writes correct, current Fish Audio code: right method names, units, and error types, instead of guessing. Here's the fastest path. ```bash theme={null} npx skills add https://docs.fish.audio ``` This installs both Fish Audio skills (a canonical copy in `.agents/skills/`, with symlinks for Claude Code and Cursor). Run `npx skills update` later to refresh them. Python (`fish-audio-sdk`) and JavaScript (`fish-audio`) — exact method signatures, sync + async, model selection, and the real exception types. Raw REST + WebSocket for any language — auth, endpoints, MessagePack/JSON/multipart rules, and the streaming protocol. [Create a key](/developer-guide/getting-started/api-key) and export it — the code your agent writes reads it from the environment: ```bash theme={null} export FISH_API_KEY="your_api_key_here" ``` Prompt in plain language — it uses the correct client, methods, and error types: "Generate speech with Fish Audio in a cloned voice and save it to a file." "Transcribe `speech.wav` with Fish Audio and print the segments." "Stream an LLM's tokens to Fish Audio TTS over the WebSocket." "Call the Fish Audio TTS REST API from Go, no SDK." ## Install options ```bash All skills theme={null} npx skills add https://docs.fish.audio ``` ```bash One skill theme={null} npx skills add https://docs.fish.audio --skill fish-audio-sdk ``` ```bash Target an agent theme={null} # claude-code, cursor, codex, ... npx skills add https://docs.fish.audio -a claude-code ``` ```bash List / inspect first theme={null} npx skills add https://docs.fish.audio --list ``` Targeting specific agents, the live-docs MCP server, skill-vs-MCP, and reading the skills before you install. ## Next steps Create a key and make your first request. Generate your first audio by hand, in any language. Voices, formats, streaming, and the direct API. Endpoints, parameters, and the OpenAPI schema. ## For autonomous agents & RAG pipelines Not a coding agent installing a skill — an autonomous agent, RAG pipeline, or crawler? Start from these low-noise, machine-readable entry points: * [llms.txt](https://docs.fish.audio/llms.txt) — curated documentation index (read this first) * [llms-full.txt](https://docs.fish.audio/llms-full.txt) — broader context across the whole site * [OpenAPI](https://docs.fish.audio/api-reference/openapi.json) — REST schemas, parameters, and examples * [AsyncAPI](https://docs.fish.audio/api-reference/asyncapi.yml) — the WebSocket streaming protocol - Base API URL: `https://api.fish.audio` - Authentication: `Authorization: Bearer ` - TTS model selection: send a required `model` header. Recommended default: `s2-pro` - Main REST endpoints: * `POST /v1/tts` * `POST /v1/asr` * `GET /model` * `POST /model` * `GET /model/{id}` * `PATCH /model/{id}` * `DELETE /model/{id}` - Real-time streaming endpoint: `wss://api.fish.audio/v1/tts/live` 1. Read [llms.txt](https://docs.fish.audio/llms.txt) for the curated documentation index. 2. Read [llms-full.txt](https://docs.fish.audio/llms-full.txt) when broad site context is needed. 3. Read [OpenAPI](https://docs.fish.audio/api-reference/openapi.json) for REST schemas, parameters, and examples. 4. Read [AsyncAPI](https://docs.fish.audio/api-reference/asyncapi.yml) for the WebSocket streaming protocol. 5. Fetch individual `.md` pages only after narrowing to a specific task. **API specs** * [OpenAPI](https://docs.fish.audio/api-reference/openapi.json) * [AsyncAPI](https://docs.fish.audio/api-reference/asyncapi.yml) * [API Introduction](https://docs.fish.audio/api-reference/introduction.md) **Auth & SDK setup** * [Python Authentication](https://docs.fish.audio/developer-guide/sdk-guide/python/authentication.md) * [JavaScript Authentication](https://docs.fish.audio/developer-guide/getting-started/api-key.md) * [Python SDK Overview](https://docs.fish.audio/api-reference/sdk/python/overview.md) * [JavaScript SDK Reference](https://docs.fish.audio/api-reference/sdk/javascript/api-reference.md) **Core product tasks** * [Text to Speech Guide](https://docs.fish.audio/features/text-to-speech.md) * [Speech to Text Guide](https://docs.fish.audio/features/speech-to-text.md) * [Creating Voice Models](https://docs.fish.audio/features/voice-cloning.md) * [Emotion Control](https://docs.fish.audio/developer-guide/core-features/emotions.md) * [Fine-grained Control](https://docs.fish.audio/developer-guide/core-features/fine-grained-control.md) **Real-time & integrations** * [WebSocket TTS Streaming](https://docs.fish.audio/api-reference/endpoint/websocket/tts-live.md) * [Real-time Streaming Best Practices](https://docs.fish.audio/developer-guide/best-practices/real-time-streaming.md) * [Realtime Streaming (SDK)](https://docs.fish.audio/features/realtime-streaming.md) * [LiveKit Integration](https://docs.fish.audio/developer-guide/integrations/livekit.md) * [Pipecat Integration](https://docs.fish.audio/developer-guide/integrations/pipecat.md) **Models, pricing & lifecycle** * [Models Overview](https://docs.fish.audio/developer-guide/models-pricing/models-overview.md) * [Choosing a Model](https://docs.fish.audio/developer-guide/models-pricing/choosing-a-model.md) * [Pricing & Rate Limits](https://docs.fish.audio/developer-guide/models-pricing/pricing-and-rate-limits.md) * [Model Deprecations](https://docs.fish.audio/developer-guide/models-pricing/deprecations.md) * **Generate speech** → Quick Start, the Text to Speech guide, and `POST /v1/tts`. * **Transcribe audio** → the Speech to Text guide and `POST /v1/asr`. * **Clone or manage voices** → Creating Voice Models and the `/model` endpoints. * **Stream audio in real time** → AsyncAPI, WebSocket TTS Streaming, and the realtime guides. * **Pick a model or estimate cost** → Models Overview and Pricing & Rate Limits. * Prefer `openapi.json` and `asyncapi.yml` for machine-readable schemas. * Append `.md` to any page URL to fetch the human-authored page as plain Markdown. * Some richer pages use interactive MDX widgets. If a fetched page contains UI or component noise, fall back to `llms.txt`, `llms-full.txt`, or the API spec files. # Brand Guidelines Source: https://docs.fish.audio/developer-guide/resources/brand Design guidelines for using Fish Audio brand assets ## Logo ### Wordmark Our preferred logo format combines the [Fish Audio Icon](#icon) with the wordmark side by side. This is the primary version of our logo and should be used whenever possible for maximum brand recognition and clarity. Fish Audio Clearspace Wordmark Fish Audio Clearspace Wordmark ### Icon Our icon features a whale composed of audio bars and sound waves, symbolizing the fusion of marine life with audio technology. This design represents our brand's commitment to natural, flowing, and powerful voice generation. The Fish Audio icon should only be used when space constraints or context make it impractical to display the full wordmark. Always prefer the wordmark with icon combination when possible. Fish Audio Clearspace Logo Fish Audio Clearspace Logo ### Avoid To maintain the integrity of our brand identity, please do not alter our logo in any of the following ways: Incorrect logo usage - distorted Incorrect logo usage - distorted Incorrect logo usage - rotated Incorrect logo usage - rotated Incorrect logo usage - wrong colors Incorrect logo usage - wrong colors Incorrect logo usage - effects Incorrect logo usage - effects ## Colors Our official brand colors consist of black and white for primary logo applications, complemented by secondary grays for subtle variations and an accent purple for visual highlights in marketing materials.
## Typography Our brand uses **Onest Semibold** in the logo wordmark. This documentation is also set in Onest, so you're experiencing our brand typography right now. [Download Onest on Google Fonts](https://fonts.google.com/specimen/Onest) ## Usage Guidelines The Fish Audio name and logos are trademarks of Hanabi AI Inc. You may freely use and redistribute our brand assets when referencing Fish Audio. By using our brand assets, you agree that we own them and that any goodwill generated by your use benefits Fish Audio. ### Do * Use our brand assets freely in your projects, applications, and content * Share our brand assets in blog posts, tutorials, documentation, and educational materials * Follow the visual guidelines shown above (spacing, colors, sizing) * Link to fish.audio when using our brand online ### Don't * Use our logo as part of your own product name or branding * Imply partnership, sponsorship, or endorsement without permission * Feature our logo more prominently than your own brand ### Questions? If you're unsure whether your use case is appropriate or need special permission, please contact us at [support@fish.audio](mailto:support@fish.audio). ## Download Assets # AI Coding Agents Source: https://docs.fish.audio/developer-guide/resources/coding-agents Install the Fish Audio skill so your coding agent writes correct SDK and API code Install the Fish Audio **agent skill**, and your coding agent — Claude Code, Cursor, Codex, and others — writes correct, current Fish Audio code: right method names, units, and error types, instead of guessing. ## Install the skill ```bash theme={null} npx skills add https://docs.fish.audio ``` This installs both Fish Audio skills into your agent (a canonical copy in `.agents/skills/`, with symlinks for Claude Code and Cursor). Run `npx skills update` later to refresh them. Python (`fish-audio-sdk`) and JavaScript (`fish-audio`) — exact method signatures and defaults, sync + async, model selection, and the real exception types. Raw REST + WebSocket for any language or edge runtime — auth, endpoints, MessagePack/JSON/multipart rules, and the streaming protocol. ### Install options ```bash All skills theme={null} npx skills add https://docs.fish.audio ``` ```bash One skill theme={null} npx skills add https://docs.fish.audio --skill fish-audio-sdk ``` ```bash Target an agent theme={null} # claude-code, cursor, codex, ... npx skills add https://docs.fish.audio -a claude-code ``` ```bash List / inspect first theme={null} npx skills add https://docs.fish.audio --list ``` Want to read them before installing? The skills are served at [/.well-known/agent-skills/index.json](https://docs.fish.audio/.well-known/agent-skills/index.json), with each skill's markdown at `/.well-known/agent-skills//SKILL.md`. ## Try it Once installed, ask your agent in plain language — it will use the correct client, methods, and error types: "Generate speech with Fish Audio in a cloned voice and save it to a file." "Transcribe `speech.wav` with Fish Audio and print the segments." "Stream an LLM's tokens to Fish Audio TTS over the WebSocket." "Call the Fish Audio TTS REST API from Go, no SDK." ## Or connect via MCP Prefer live documentation search inside your editor over a self-contained skill file? Connect the Fish Audio MCP server, which serves the latest docs to your agent. ```bash theme={null} claude mcp add --transport http fish-audio --scope project https://docs.fish.audio/mcp ``` This writes a `.mcp.json` in your project root. Verify with `claude mcp list` (you should see `fish-audio`), then ask "What Fish Audio TTS models are available?" * **`--scope project`** (recommended): config in `.mcp.json`, version-controlled and shared with your team. * **`--scope user`**: global across your projects, private to your account. * **`--scope local`** (default): project-specific and private to you. Open the command palette (`Cmd/Ctrl+Shift+P`) → "Open MCP settings" → "Add custom MCP", and add: ```json theme={null} { "mcpServers": { "fish-audio": { "url": "https://docs.fish.audio/mcp" } } } ``` Save and reload Cursor, then ask "What Fish Audio TTS models are available?" Go to `Settings → Cascade → MCP Servers → View raw config` (`~/.codeium/windsurf/mcp_config.json`) and add: ```json theme={null} { "mcpServers": { "fish-audio": { "url": "https://docs.fish.audio/mcp" } } } ``` Save, refresh, then in Cascade ask "Search Fish Audio docs for TTS API usage." **Skill vs MCP:** the skill is a self-contained instruction file that works offline after install; MCP fetches the latest docs live. You can use both. This site also exposes [llms.txt](https://docs.fish.audio/llms.txt) and [llms-full.txt](https://docs.fish.audio/llms-full.txt) for agents that fetch docs directly. ## Next steps Create a key and make your first request. Voices, formats, streaming, and the direct API. Endpoints, parameters, and the OpenAPI schema. Status codes, retries, and SDK exception handling. ## Support * **Technical support**: [support@fish.audio](mailto:support@fish.audio) * **Issues**: [GitHub](https://github.com/fishaudio) * **Community**: [Discord](https://discord.gg/dF9Db2Tt3Y) # Batch-transcribe files with a language hint Source: https://docs.fish.audio/developer-guide/sdk-guide/cookbook/batch-transcribe-with-language-hint Loop over local audio files and transcribe each with an explicit language, collecting text and duration per file ## Prerequisites Sign up for a free Fish Audio account to get started with our API. 1. Go to [fish.audio/auth/signup](https://fish.audio/auth/signup) 2. Fill in your details to create an account, complete steps to verify your account. 3. Log in to your account and navigate to the [API section](https://fish.audio/app/api-keys) Once you have an account, you'll need an API key to authenticate your requests. 1. Log in to your [Fish Audio Dashboard](https://fish.audio/app/api-keys/) 2. Navigate to the API Keys section 3. Click "Create New Key" and give it a descriptive name, set a expiration if desired 4. Copy your key and store it securely Keep your API key secret! Never commit it to version control or share it publicly. ## Recipe Read each file's bytes from disk and pass them to [`asr.transcribe()`](/api-reference/sdk/python/resources#transcribe) with an explicit `language`. A language hint is more reliable than auto-detection when you already know the source language, especially for phonetically similar languages. Collect one result row per file as you go. ```python Synchronous theme={null} from fishaudio import FishAudio client = FishAudio() paths = ["speech.wav"] # add more file paths here language = "en" results = [] for path in paths: with open(path, "rb") as f: audio = f.read() transcript = client.asr.transcribe(audio=audio, language=language) results.append({ "file": path, "text": transcript.text, "duration": transcript.duration, # seconds }) for row in results: print(f"{row['file']} ({row['duration']:.1f}s): {row['text']}") ``` ```python Asynchronous theme={null} import asyncio from fishaudio import AsyncFishAudio async def main(): async with AsyncFishAudio() as client: paths = ["speech.wav"] # add more file paths here language = "en" results = [] for path in paths: with open(path, "rb") as f: audio = f.read() transcript = await client.asr.transcribe(audio=audio, language=language) results.append({ "file": path, "text": transcript.text, "duration": transcript.duration, # seconds }) for row in results: print(f"{row['file']} ({row['duration']:.1f}s): {row['text']}") asyncio.run(main()) ``` ```javascript JavaScript theme={null} import { FishAudioClient } from "fish-audio"; import { readFile } from "fs/promises"; const client = new FishAudioClient({ apiKey: process.env.FISH_API_KEY }); const paths = ["speech.wav"]; // add more file paths here const language = "en"; const results = []; for (const path of paths) { const audio = new File([await readFile(path)], path); const transcript = await client.speechToText.convert({ audio, language }); results.push({ file: path, text: transcript.text, duration: transcript.duration, // seconds }); } for (const row of results) { console.log(`${row.file} (${row.duration.toFixed(1)}s): ${row.text}`); } ``` Each call returns an [`ASRResponse`](/api-reference/sdk/python/types#asrresponse-objects) with `.text`, a `.duration` in seconds, and per-phrase `.segments`. The loop keeps files independent, so one bad file does not block the rest of the batch. Auto-detection (omit `language`) works well, but passing an explicit `language` improves accuracy for similar-sounding languages. Use one `language` per batch — split mixed-language files into separate lists. ## Related * [Speech-to-Text guide](/features/speech-to-text) * [Instant voice cloning](/developer-guide/sdk-guide/cookbook/instant-voice-cloning) # Clone a voice and wait until it is ready Source: https://docs.fish.audio/developer-guide/sdk-guide/cookbook/clone-and-wait-until-ready Create a persistent voice from a reference clip, poll until training finishes, then synthesize with it A persistent voice is trained asynchronously: `voices.create()` returns immediately with a voice whose `state` is `created` or `training`. Before you can synthesize with it, you need to wait until its `state` becomes `trained`. This recipe creates a voice from `reference.wav`, polls [`voices.get()`](/api-reference/sdk/python/resources#get) until training finishes (with a timeout), then synthesizes with `reference_id`. ## Prerequisites Sign up for a free Fish Audio account to get started with our API. 1. Go to [fish.audio/auth/signup](https://fish.audio/auth/signup) 2. Fill in your details to create an account, complete steps to verify your account. 3. Log in to your account and navigate to the [API section](https://fish.audio/app/api-keys) Once you have an account, you'll need an API key to authenticate your requests. 1. Log in to your [Fish Audio Dashboard](https://fish.audio/app/api-keys/) 2. Navigate to the API Keys section 3. Click "Create New Key" and give it a descriptive name, set a expiration if desired 4. Copy your key and store it securely Keep your API key secret! Never commit it to version control or share it publicly. ## Recipe Poll `voices.get(voice.id).state` on an interval, stopping when it reaches `trained` (or raising if it `failed` or the timeout elapses). Then pass the voice id as `reference_id` on `convert()`. ```python Synchronous theme={null} import time from fishaudio import FishAudio from fishaudio.utils import save client = FishAudio() # 1. Create a persistent voice from a reference clip. with open("reference.wav", "rb") as f: voice = client.voices.create(title="My Voice", voices=[f.read()]) # 2. Poll until the voice finishes training. deadline = time.time() + 300 # 5-minute timeout while voice.state != "trained": if voice.state == "failed": raise RuntimeError(f"Voice {voice.id} failed to train") if time.time() > deadline: raise TimeoutError(f"Voice {voice.id} not ready (state={voice.state})") time.sleep(5) voice = client.voices.get(voice.id) # 3. Synthesize with the trained voice. audio = client.tts.convert( text="My voice is ready to use.", reference_id=voice.id, ) save(audio, "out.mp3") ``` ```python Asynchronous theme={null} import asyncio from fishaudio import AsyncFishAudio from fishaudio.utils import save async def main(): async with AsyncFishAudio() as client: # 1. Create a persistent voice from a reference clip. with open("reference.wav", "rb") as f: voice = await client.voices.create(title="My Voice", voices=[f.read()]) # 2. Poll until the voice finishes training. deadline = asyncio.get_event_loop().time() + 300 # 5-minute timeout while voice.state != "trained": if voice.state == "failed": raise RuntimeError(f"Voice {voice.id} failed to train") if asyncio.get_event_loop().time() > deadline: raise TimeoutError(f"Voice {voice.id} not ready (state={voice.state})") await asyncio.sleep(5) voice = await client.voices.get(voice.id) # 3. Synthesize with the trained voice. audio = await client.tts.convert( text="My voice is ready to use.", reference_id=voice.id, ) save(audio, "out.mp3") asyncio.run(main()) ``` ```javascript JavaScript theme={null} import { readFile, writeFile } from "fs/promises"; import { FishAudioClient } from "fish-audio"; const client = new FishAudioClient({ apiKey: process.env.FISH_API_KEY }); // 1. Create a persistent voice from a reference clip. const sample = new File([await readFile("reference.wav")], "reference.wav"); let voice = await client.voices.ivc.create({ title: "My Voice", visibility: "private", voices: [sample], }); // 2. Poll until the voice finishes training. const deadline = Date.now() + 300_000; // 5-minute timeout while (voice.state !== "trained") { if (voice.state === "failed") { throw new Error(`Voice ${voice._id} failed to train`); } if (Date.now() > deadline) { throw new Error(`Voice ${voice._id} not ready (state=${voice.state})`); } await new Promise((resolve) => setTimeout(resolve, 5000)); voice = await client.voices.get(voice._id); } // 3. Synthesize with the trained voice. const stream = await client.textToSpeech.convert( { text: "My voice is ready to use.", reference_id: voice._id }, "s2-pro", ); const chunks = []; for await (const chunk of stream) chunks.push(Buffer.from(chunk)); await writeFile("out.mp3", Buffer.concat(chunks)); ``` A voice moves through `created` → `training` → `trained`, or ends in `failed`. Always handle `failed` and the timeout so a stuck voice cannot loop forever. Training a persistent voice takes time, so only create one when you will reuse the voice across many requests. For one-off synthesis, skip the wait entirely and pass a `ReferenceAudio` inline — see [Instant voice cloning](/developer-guide/sdk-guide/cookbook/instant-voice-cloning). ## Related * [Instant voice cloning](/developer-guide/sdk-guide/cookbook/instant-voice-cloning) * [Voice Cloning guide](/features/voice-cloning) * [Voices API reference](/api-reference/sdk/python/resources#voices) # Discover and reuse a Voice Library voice Source: https://docs.fish.audio/developer-guide/sdk-guide/cookbook/discover-library-voice Search the public Voice Library by title, pick a result, and synthesize speech with it as your reference_id ## Prerequisites Sign up for a free Fish Audio account to get started with our API. 1. Go to [fish.audio/auth/signup](https://fish.audio/auth/signup) 2. Fill in your details to create an account, complete steps to verify your account. 3. Log in to your account and navigate to the [API section](https://fish.audio/app/api-keys) Once you have an account, you'll need an API key to authenticate your requests. 1. Log in to your [Fish Audio Dashboard](https://fish.audio/app/api-keys/) 2. Navigate to the API Keys section 3. Click "Create New Key" and give it a descriptive name, set a expiration if desired 4. Copy your key and store it securely Keep your API key secret! Never commit it to version control or share it publicly. ## Recipe Set `self_only=False` on [`voices.list()`](/api-reference/sdk/python/resources#list) to search the public [Voice Library](/features/manage-voices) instead of only your own models. The response carries `total` (matches across all pages) and `items` (this page). Pick a result's `id` and pass it straight to [`tts.convert()`](/api-reference/sdk/python/resources#convert) as `reference_id` — no cloning, no model to manage. ```python Python theme={null} from fishaudio import FishAudio from fishaudio.utils import save client = FishAudio() # reads FISH_API_KEY # Search the public library by title (not just your own voices) page = client.voices.list(title="narration", self_only=False, page_size=10) print(f"{page.total} public voices match") # Pick the first result; fall back to a known id if the search is empty reference_id = "" for voice in page.items: print(voice.id, voice.title, voice.languages) reference_id = reference_id if reference_id != "" else voice.id # Synthesize with the discovered voice as the reference audio = client.tts.convert( text="Speaking with a voice I found in the public library.", reference_id=reference_id, ) save(audio, "out.mp3") ``` ```javascript JavaScript theme={null} import { FishAudioClient } from "fish-audio"; import { writeFile } from "fs/promises"; const client = new FishAudioClient({ apiKey: process.env.FISH_API_KEY }); // Search the public Voice Library by title (not just your own voices) const page = await client.voices.search({ title: "narration", page_size: 10 }); console.log(`${page.total} public voices match`); // Pick the first result; fall back to a known id if the search is empty let referenceId = ""; for (const voice of page.items) { console.log(voice._id, voice.title, voice.languages); referenceId = referenceId !== "" ? referenceId : voice._id; } // Synthesize with the discovered voice as the reference const stream = await client.textToSpeech.convert( { text: "Speaking with a voice I found in the public library.", reference_id: referenceId, format: "mp3", }, "s2-pro" ); const chunks = []; for await (const chunk of stream) chunks.push(Buffer.from(chunk)); await writeFile("out.mp3", Buffer.concat(chunks)); ``` `page.total` is the full match count, so `total > len(page.items)` tells you there are more pages — bump `page_number` to walk them. Any public voice `id` is a ready-to-use `reference_id`; nothing is saved to your account. You can hit the same endpoint directly: ```bash theme={null} curl "https://api.fish.audio/model?title=narration&page_size=10" \ --header "Authorization: Bearer $FISH_API_KEY" # Response: { "total": 128, "items": [ { "_id": "...", "title": "...", ... } ] } ``` Title search is fuzzy and ranked by usage, so the top result is usually the most popular match. Add `language=["en"]` to narrow by spoken language, or raise `page_size` and page with `page_number` to scan deeper. ## Related * [Manage Voices](/features/manage-voices) * [Instant voice cloning](/developer-guide/sdk-guide/cookbook/instant-voice-cloning) * [Python reference: voices](/api-reference/sdk/python/resources) # Instant voice cloning Source: https://docs.fish.audio/developer-guide/sdk-guide/cookbook/instant-voice-cloning Clone a voice on the fly from a short reference clip, with no model to manage ## Prerequisites Sign up for a free Fish Audio account to get started with our API. 1. Go to [fish.audio/auth/signup](https://fish.audio/auth/signup) 2. Fill in your details to create an account, complete steps to verify your account. 3. Log in to your account and navigate to the [API section](https://fish.audio/app/api-keys) Once you have an account, you'll need an API key to authenticate your requests. 1. Log in to your [Fish Audio Dashboard](https://fish.audio/app/api-keys/) 2. Navigate to the API Keys section 3. Click "Create New Key" and give it a descriptive name, set a expiration if desired 4. Copy your key and store it securely Keep your API key secret! Never commit it to version control or share it publicly. ## Recipe Pass a [`ReferenceAudio`](/api-reference/sdk/python/types#referenceaudio-objects) (raw audio bytes + an exact transcript) on the `convert` call. Nothing is saved server-side — the clone applies to that request only. ```python Synchronous theme={null} from fishaudio import FishAudio from fishaudio.types import ReferenceAudio from fishaudio.utils import save client = FishAudio() with open("reference.wav", "rb") as f: audio = client.tts.convert( text="This sentence is spoken in the cloned voice.", references=[ReferenceAudio( audio=f.read(), text="Exact transcript of what is said in reference.wav.", )], ) save(audio, "cloned.mp3") ``` ```python Asynchronous theme={null} import asyncio from fishaudio import AsyncFishAudio from fishaudio.types import ReferenceAudio from fishaudio.utils import save async def main(): async with AsyncFishAudio() as client: with open("reference.wav", "rb") as f: audio = await client.tts.convert( text="This sentence is spoken in the cloned voice.", references=[ReferenceAudio( audio=f.read(), text="Exact transcript of what is said in reference.wav.", )], ) save(audio, "cloned.mp3") asyncio.run(main()) ``` ```javascript JavaScript theme={null} import { FishAudioClient } from "fish-audio"; import { readFile, writeFile } from "fs/promises"; const client = new FishAudioClient({ apiKey: process.env.FISH_API_KEY }); const reference = new File( [await readFile("reference.wav")], "reference.wav", ); const stream = await client.textToSpeech.convert( { text: "This sentence is spoken in the cloned voice.", references: [ { audio: reference, text: "Exact transcript of what is said in reference.wav.", }, ], format: "mp3", }, "s2-pro", ); const chunks = []; for await (const chunk of stream) chunks.push(Buffer.from(chunk)); await writeFile("cloned.mp3", Buffer.concat(chunks)); ``` Use 10–30 s of clean speech, and make `text` match the audio exactly (including punctuation) for the best prosody. ## Reuse a voice across many requests If you'll use the voice repeatedly, create a persistent model once and pass its id as `reference_id` — see the [Voice Cloning guide](/features/voice-cloning). ```python theme={null} with open("sample.wav", "rb") as f: voice = client.voices.create(title="My Voice", voices=[f.read()]) audio = client.tts.convert(text="Reusing my saved voice.", reference_id=voice.id) ``` ## Related * [Voice Cloning guide](/features/voice-cloning) * [Stream TTS to a file](/developer-guide/sdk-guide/cookbook/streaming-to-file) # One-shot vs persistent cloning: pick the right approach Source: https://docs.fish.audio/developer-guide/sdk-guide/cookbook/oneshot-vs-persistent-cloning Choose between instant per-request cloning and a saved, reusable voice model ## Prerequisites Sign up for a free Fish Audio account to get started with our API. 1. Go to [fish.audio/auth/signup](https://fish.audio/auth/signup) 2. Fill in your details to create an account, complete steps to verify your account. 3. Log in to your account and navigate to the [API section](https://fish.audio/app/api-keys) Once you have an account, you'll need an API key to authenticate your requests. 1. Log in to your [Fish Audio Dashboard](https://fish.audio/app/api-keys/) 2. Navigate to the API Keys section 3. Click "Create New Key" and give it a descriptive name, set a expiration if desired 4. Copy your key and store it securely Keep your API key secret! Never commit it to version control or share it publicly. ## Recipe There are two ways to clone a voice. Pick by how often you'll reuse it: * **One-shot (instant)** — pass a [`ReferenceAudio`](/api-reference/sdk/python/types#referenceaudio-objects) (raw bytes + exact transcript) on each `convert` call. Nothing is stored server-side; the clone lives only for that request. * **Persistent** — call `voices.create` once to train a model, then reuse its id as `reference_id` on every request. No reference upload per call, and the same voice is shared across processes. Start with one-shot. Below, a single reference clip is cloned inline with no model to manage: ```python Synchronous theme={null} from fishaudio import FishAudio from fishaudio.types import ReferenceAudio from fishaudio.utils import save client = FishAudio() with open("reference.wav", "rb") as f: audio = client.tts.convert( text="This line is spoken in the cloned voice, no model required.", references=[ReferenceAudio( audio=f.read(), text="Exact transcript of what is said in reference.wav.", )], ) save(audio, "oneshot.mp3") ``` ```python Asynchronous theme={null} import asyncio from fishaudio import AsyncFishAudio from fishaudio.types import ReferenceAudio from fishaudio.utils import save async def main(): async with AsyncFishAudio() as client: with open("reference.wav", "rb") as f: audio = await client.tts.convert( text="This line is spoken in the cloned voice, no model required.", references=[ReferenceAudio( audio=f.read(), text="Exact transcript of what is said in reference.wav.", )], ) save(audio, "oneshot.mp3") asyncio.run(main()) ``` ```javascript JavaScript theme={null} import { FishAudioClient } from "fish-audio"; import { readFile, writeFile } from "fs/promises"; const client = new FishAudioClient({ apiKey: process.env.FISH_API_KEY }); // One-shot: clone inline by sending the reference bytes + exact transcript. // Nothing is stored server-side; the clone lives only for this request. const reference = await readFile("reference.wav"); const stream = await client.textToSpeech.convert( { text: "This line is spoken in the cloned voice, no model required.", references: [ { audio: new File([reference], "reference.wav"), text: "Exact transcript of what is said in reference.wav.", }, ], format: "mp3", }, "s2-pro" ); const chunks = []; for await (const chunk of stream) chunks.push(Buffer.from(chunk)); await writeFile("oneshot.mp3", Buffer.concat(chunks)); ``` One-shot re-sends the reference bytes on every request, so it's ideal for one-off or rarely-repeated voices. Once a voice is used more than a handful of times, switch to a persistent model to skip the per-call upload. ## Train a persistent voice once, reuse forever Call [`voices.create`](/api-reference/sdk/python/resources#create) to train a model, then pass `voice.id` as `reference_id`. The same id works from any process and across SDK and REST. ```python theme={null} with open("reference.wav", "rb") as f: voice = client.voices.create(title="My Narrator", voices=[f.read()]) # reuse the same id on every later request — no reference upload audio = client.tts.convert( text="Reusing my saved voice across many requests.", reference_id=voice.id, ) save(audio, "persistent.mp3") ``` Already have a trained voice id? Skip training and pass it directly: ```python theme={null} audio = client.tts.convert(text="Hello again.", reference_id="") ``` ## Which to choose | | One-shot | Persistent | | ------------------ | ------------------------------ | --------------------------------------------------- | | Setup | None | One `voices.create` call | | Per request | Re-uploads reference bytes | Sends only `reference_id` | | Stored server-side | No | Yes (manage with `voices.update` / `voices.delete`) | | Best for | One-off or experimental clones | Voices reused many times or across services | For either path, give the reference 10–30 s of clean speech and make the transcript match the audio exactly (including punctuation) for the best prosody. ## Related * [Instant voice cloning](/developer-guide/sdk-guide/cookbook/instant-voice-cloning) * [Voice Cloning guide](/features/voice-cloning) * [Manage voices](/features/manage-voices) # Realtime: LLM tokens → speech Source: https://docs.fish.audio/developer-guide/sdk-guide/cookbook/realtime-llm-to-speech Pipe a streaming LLM response straight into speech over a WebSocket as tokens arrive ## Prerequisites Sign up for a free Fish Audio account to get started with our API. 1. Go to [fish.audio/auth/signup](https://fish.audio/auth/signup) 2. Fill in your details to create an account, complete steps to verify your account. 3. Log in to your account and navigate to the [API section](https://fish.audio/app/api-keys) Once you have an account, you'll need an API key to authenticate your requests. 1. Log in to your [Fish Audio Dashboard](https://fish.audio/app/api-keys/) 2. Navigate to the API Keys section 3. Click "Create New Key" and give it a descriptive name, set a expiration if desired 4. Copy your key and store it securely Keep your API key secret! Never commit it to version control or share it publicly. ## Recipe [`stream_websocket()`](/api-reference/sdk/python/resources#stream_websocket) takes an iterable of text chunks and yields audio chunks in real time. Feed it your LLM's token stream and play or forward the audio as it's produced. ```python Synchronous theme={null} from fishaudio import FishAudio from fishaudio.utils import play client = FishAudio() def llm_tokens(): # Replace with your real streaming LLM call for token in ["The ", "first ", "move ", "sets ", "everything ", "in ", "motion."]: yield token audio_stream = client.tts.stream_websocket(llm_tokens(), reference_id="") play(audio_stream) # or: for chunk in audio_stream: send_to_client(chunk) ``` ```python Asynchronous theme={null} import asyncio from fishaudio import AsyncFishAudio async def llm_tokens(): for token in ["The ", "first ", "move ", "sets ", "everything ", "in ", "motion."]: yield token async def main(): async with AsyncFishAudio() as client: audio_stream = client.tts.stream_websocket(llm_tokens()) with open("out.mp3", "wb") as f: async for chunk in audio_stream: f.write(chunk) asyncio.run(main()) ``` ## Force generation at a boundary By default the engine buffers text until it has enough for natural prosody. Yield a [`FlushEvent`](/api-reference/sdk/python/types#flushevent-objects) to force synthesis of what's buffered — useful for turn-taking in a conversation: ```python theme={null} from fishaudio.types import TextEvent, FlushEvent def turns(): yield TextEvent(text="Are you ready?") yield FlushEvent() # speak the question now yield TextEvent(text="Let's begin.") ``` The SDK sends the start/stop frames for you — you only supply text and optional flushes. Errors mid-stream surface as `WebSocketError`. Reconnect with a fresh call rather than retrying on the same socket. ## Related * [Realtime WebSocket guide](/features/realtime-streaming) * [Errors & Retries](/developer-guide/sdk-guide/python/errors) # Stream TTS to a file Source: https://docs.fish.audio/developer-guide/sdk-guide/cookbook/streaming-to-file Generate long audio and write it to disk chunk-by-chunk, without buffering it all in memory ## Prerequisites Sign up for a free Fish Audio account to get started with our API. 1. Go to [fish.audio/auth/signup](https://fish.audio/auth/signup) 2. Fill in your details to create an account, complete steps to verify your account. 3. Log in to your account and navigate to the [API section](https://fish.audio/app/api-keys) Once you have an account, you'll need an API key to authenticate your requests. 1. Log in to your [Fish Audio Dashboard](https://fish.audio/app/api-keys/) 2. Navigate to the API Keys section 3. Click "Create New Key" and give it a descriptive name, set a expiration if desired 4. Copy your key and store it securely Keep your API key secret! Never commit it to version control or share it publicly. ## Recipe For long text, use [`stream()`](/api-reference/sdk/python/resources#stream) and write each chunk as it arrives instead of holding the whole file in memory. ```python Synchronous theme={null} from fishaudio import FishAudio client = FishAudio() with open("output.mp3", "wb") as f: for chunk in client.tts.stream(text="A very long passage of text..."): f.write(chunk) ``` ```python Asynchronous theme={null} import asyncio from fishaudio import AsyncFishAudio async def main(): async with AsyncFishAudio() as client: audio_stream = await client.tts.stream(text="A very long passage of text...") with open("output.mp3", "wb") as f: async for chunk in audio_stream: f.write(chunk) asyncio.run(main()) ``` ```javascript JavaScript theme={null} import { FishAudioClient } from "fish-audio"; import { createWriteStream } from "fs"; const client = new FishAudioClient({ apiKey: process.env.FISH_API_KEY }); // convert() returns a ReadableStream. Iterate it and write each // chunk as it arrives, so you never hold the whole file in memory. const stream = await client.textToSpeech.convert( { text: "A very long passage of text...", format: "mp3" }, "s2-pro" ); const file = createWriteStream("output.mp3"); for await (const chunk of stream) { file.write(Buffer.from(chunk)); } file.end(); ``` ## Collect instead of iterate If you just want the full bytes, call `.collect()`: ```python theme={null} audio = client.tts.stream(text="Hello!").collect() # -> bytes ``` `convert()` already returns the complete audio as `bytes` — reach for `stream()` when you want to start writing/forwarding bytes before generation finishes, or to avoid buffering large files. ## Related * [Text-to-Speech guide](/features/text-to-speech) * [Realtime: LLM tokens → speech](/developer-guide/sdk-guide/cookbook/realtime-llm-to-speech) # Telephony-grade audio (8 kHz) for IVR and phone Source: https://docs.fish.audio/developer-guide/sdk-guide/cookbook/telephony-8khz-audio Generate 8 kHz mono WAV/PCM that matches the narrowband sample rate phone networks expect for IVR and call-center playback ## Prerequisites Sign up for a free Fish Audio account to get started with our API. 1. Go to [fish.audio/auth/signup](https://fish.audio/auth/signup) 2. Fill in your details to create an account, complete steps to verify your account. 3. Log in to your account and navigate to the [API section](https://fish.audio/app/api-keys) Once you have an account, you'll need an API key to authenticate your requests. 1. Log in to your [Fish Audio Dashboard](https://fish.audio/app/api-keys/) 2. Navigate to the API Keys section 3. Click "Create New Key" and give it a descriptive name, set a expiration if desired 4. Copy your key and store it securely Keep your API key secret! Never commit it to version control or share it publicly. ## Recipe Phone networks carry narrowband audio at 8 kHz. Generating at a higher rate just forces the carrier to downsample on the way through — wasting bandwidth and often softening the result. Synthesize at 8 kHz directly and the bytes are ready to hand to your IVR or SIP stack. Set the sample rate on [`TTSConfig`](/api-reference/sdk/python/types#ttsconfig-objects) (it is not a top-level argument) and write the WAV to disk. ```python Synchronous theme={null} from fishaudio import FishAudio from fishaudio.types import TTSConfig from fishaudio.utils import save client = FishAudio() audio = client.tts.convert( text="Thank you for calling. Press one to speak with an agent.", config=TTSConfig(format="wav", sample_rate=8000), ) save(audio, "out.wav") ``` ```python Asynchronous theme={null} import asyncio from fishaudio import AsyncFishAudio from fishaudio.types import TTSConfig from fishaudio.utils import save async def main(): async with AsyncFishAudio() as client: audio = await client.tts.convert( text="Thank you for calling. Press one to speak with an agent.", config=TTSConfig(format="wav", sample_rate=8000), ) save(audio, "out.wav") asyncio.run(main()) ``` ```javascript JavaScript theme={null} import { FishAudioClient } from "fish-audio"; import { writeFile } from "fs/promises"; const client = new FishAudioClient({ apiKey: process.env.FISH_API_KEY }); const stream = await client.textToSpeech.convert( { text: "Thank you for calling. Press one to speak with an agent.", format: "wav", sample_rate: 8000, }, "s2-pro" ); const chunks = []; for await (const chunk of stream) chunks.push(Buffer.from(chunk)); await writeFile("out.wav", Buffer.concat(chunks)); ``` The output is a mono 8 kHz WAV — the standard for G.711 PCM telephony. For a headerless stream to feed straight into a SIP or RTP pipeline, switch to raw PCM with `format="pcm"`; the sample rate stays on `TTSConfig`. ```python theme={null} audio = client.tts.convert( text="Thank you for calling. Press one to speak with an agent.", config=TTSConfig(format="pcm", sample_rate=8000), ) ``` ```bash API (curl) theme={null} curl --request POST https://api.fish.audio/v1/tts \ --header "Authorization: Bearer $FISH_API_KEY" \ --header "Content-Type: application/json" \ --header "model: s2-pro" \ --data '{ "text": "Thank you for calling. Press one to speak with an agent.", "format": "wav", "sample_rate": 8000 }' \ --output out.wav ``` 8 kHz discards everything above \~4 kHz, so plosives and sibilance lose detail. Keep prompts short and articulate, and reserve higher sample rates (16/24 kHz) for VoIP or recordings that never touch the legacy phone network. ## Related * [Text-to-Speech guide](/features/text-to-speech) * [Stream TTS to a file](/developer-guide/sdk-guide/cookbook/streaming-to-file) # Transcribe audio to SRT/VTT captions Source: https://docs.fish.audio/developer-guide/sdk-guide/cookbook/transcribe-to-captions Transcribe audio with timestamps and write valid SRT and WebVTT caption files from the segments ## Prerequisites Sign up for a free Fish Audio account to get started with our API. 1. Go to [fish.audio/auth/signup](https://fish.audio/auth/signup) 2. Fill in your details to create an account, complete steps to verify your account. 3. Log in to your account and navigate to the [API section](https://fish.audio/app/api-keys) Once you have an account, you'll need an API key to authenticate your requests. 1. Log in to your [Fish Audio Dashboard](https://fish.audio/app/api-keys/) 2. Navigate to the API Keys section 3. Click "Create New Key" and give it a descriptive name, set a expiration if desired 4. Copy your key and store it securely Keep your API key secret! Never commit it to version control or share it publicly. ## Recipe Call [`asr.transcribe()`](/api-reference/sdk/python/resources#transcribe) with `include_timestamps=True`, then turn each [`ASRSegment`](/api-reference/sdk/python/types#asrsegment-objects) into a numbered cue. Segment `start` / `end` are in **seconds**, so the only real work is formatting them — SRT wants `HH:MM:SS,mmm` (comma), WebVTT wants `HH:MM:SS.mmm` (dot). ```python Python theme={null} from fishaudio import FishAudio client = FishAudio() def to_srt_timestamp(seconds: float) -> str: """Format a time in seconds as an SRT timestamp: HH:MM:SS,mmm.""" millis = round(seconds * 1000) hours, millis = divmod(millis, 3_600_000) minutes, millis = divmod(millis, 60_000) secs, millis = divmod(millis, 1000) return f"{hours:02d}:{minutes:02d}:{secs:02d},{millis:03d}" with open("speech.wav", "rb") as f: result = client.asr.transcribe(audio=f.read(), include_timestamps=True) # SRT: 1-based index, comma decimal separator, blank line between cues. with open("captions.srt", "w", encoding="utf-8") as srt: for i, segment in enumerate(result.segments, start=1): start = to_srt_timestamp(segment.start) end = to_srt_timestamp(segment.end) srt.write(f"{i}\n{start} --> {end}\n{segment.text.strip()}\n\n") # WebVTT: same cues, "WEBVTT" header, dot decimal separator. with open("captions.vtt", "w", encoding="utf-8") as vtt: vtt.write("WEBVTT\n\n") for segment in result.segments: start = to_srt_timestamp(segment.start).replace(",", ".") end = to_srt_timestamp(segment.end).replace(",", ".") vtt.write(f"{start} --> {end}\n{segment.text.strip()}\n\n") print(f"Wrote {len(result.segments)} cues to captions.srt and captions.vtt") ``` ```javascript JavaScript theme={null} import { FishAudioClient } from "fish-audio"; import { readFile, writeFile } from "fs/promises"; const client = new FishAudioClient({ apiKey: process.env.FISH_API_KEY }); // Format a time in seconds as an SRT timestamp: HH:MM:SS,mmm. function toSrtTimestamp(seconds) { let millis = Math.round(seconds * 1000); const hours = Math.floor(millis / 3_600_000); millis -= hours * 3_600_000; const minutes = Math.floor(millis / 60_000); millis -= minutes * 60_000; const secs = Math.floor(millis / 1000); millis -= secs * 1000; const pad = (n, width) => String(n).padStart(width, "0"); return `${pad(hours, 2)}:${pad(minutes, 2)}:${pad(secs, 2)},${pad(millis, 3)}`; } const result = await client.speechToText.convert({ audio: new File([await readFile("speech.wav")], "speech.wav"), language: "en", ignore_timestamps: false, }); // SRT: 1-based index, comma decimal separator, blank line between cues. const cues = result.segments.map((segment, i) => { const start = toSrtTimestamp(segment.start); const end = toSrtTimestamp(segment.end); return `${i + 1}\n${start} --> ${end}\n${segment.text.trim()}\n`; }); await writeFile("captions.srt", cues.join("\n"), "utf-8"); console.log(`Wrote ${result.segments.length} cues to captions.srt`); ``` Both files share one timestamp helper — WebVTT is just the SRT formatting with `,` swapped for `.`, so there is no second formatter to keep in sync. Pass `language=` (for example `"en"` or `"zh"`) when you know it — explicit language selection sharpens segment boundaries, which keeps your cue timing tight. ## Related * [Speech-to-Text guide](/features/speech-to-text) * [ASR Types Reference](/api-reference/sdk/python/types#asr) # Build a voice agent loop: speech in, reply, speech out Source: https://docs.fish.audio/developer-guide/sdk-guide/cookbook/voice-agent-loop Transcribe an utterance, generate a reply with your own LLM, and stream that reply back out as speech ## Prerequisites Sign up for a free Fish Audio account to get started with our API. 1. Go to [fish.audio/auth/signup](https://fish.audio/auth/signup) 2. Fill in your details to create an account, complete steps to verify your account. 3. Log in to your account and navigate to the [API section](https://fish.audio/app/api-keys) Once you have an account, you'll need an API key to authenticate your requests. 1. Log in to your [Fish Audio Dashboard](https://fish.audio/app/api-keys/) 2. Navigate to the API Keys section 3. Click "Create New Key" and give it a descriptive name, set a expiration if desired 4. Copy your key and store it securely Keep your API key secret! Never commit it to version control or share it publicly. ## Recipe A voice agent is three stages chained together: [`asr.transcribe()`](/api-reference/sdk/python/resources#transcribe) turns the caller's audio into text, your own LLM turns that text into a reply, and [`tts.stream()`](/api-reference/sdk/python/resources#stream) turns the reply back into speech. The transcript and the reply are just strings, so the only Fish Audio-specific parts are the first and last calls. Streaming the reply lets you start writing (or forwarding) audio before the whole sentence is synthesized. ```python Synchronous theme={null} from fishaudio import FishAudio from fishaudio.utils import save client = FishAudio() def reply_from_llm(text: str) -> str: # ---- PLACEHOLDER ---- # Call your own LLM here and return its reply as a string. # e.g. return openai_client.chat.completions.create(...).choices[0].message.content return f"You said: {text}. How can I help?" def voice_agent_turn(audio_path: str, out_path: str) -> str: with open(audio_path, "rb") as f: heard = client.asr.transcribe(audio=f.read()) reply = reply_from_llm(heard.text) audio_stream = client.tts.stream(text=reply, reference_id="") save(audio_stream, out_path) # writes chunks as they arrive return reply reply = voice_agent_turn("speech.wav", "reply.mp3") print("Agent:", reply) ``` ```python Asynchronous theme={null} import asyncio from fishaudio import AsyncFishAudio from fishaudio.utils import save def reply_from_llm(text: str) -> str: # ---- PLACEHOLDER ---- # Call your own LLM here and return its reply as a string. return f"You said: {text}. How can I help?" async def main(): async with AsyncFishAudio() as client: with open("speech.wav", "rb") as f: heard = await client.asr.transcribe(audio=f.read()) reply = reply_from_llm(heard.text) audio_stream = await client.tts.stream(text=reply, reference_id="") with open("reply.mp3", "wb") as out: async for chunk in audio_stream: out.write(chunk) print("Agent:", reply) asyncio.run(main()) ``` ```javascript JavaScript theme={null} import { FishAudioClient } from "fish-audio"; import { readFile, writeFile } from "fs/promises"; const client = new FishAudioClient({ apiKey: process.env.FISH_API_KEY }); function replyFromLlm(text) { // ---- PLACEHOLDER ---- // Call your own LLM here and return its reply as a string. // e.g. return openaiClient.chat.completions.create(...).choices[0].message.content return `You said: ${text}. How can I help?`; } async function voiceAgentTurn(audioPath, outPath) { const heard = await client.speechToText.convert({ audio: new File([await readFile(audioPath)], audioPath), language: "en", }); const reply = replyFromLlm(heard.text); const stream = await client.textToSpeech.convert( { text: reply, reference_id: "", format: "mp3" }, "s2-pro" ); const chunks = []; for await (const chunk of stream) chunks.push(Buffer.from(chunk)); await writeFile(outPath, Buffer.concat(chunks)); return reply; } const reply = await voiceAgentTurn("speech.wav", "reply.mp3"); console.log("Agent:", reply); ``` `heard` is an [`ASRResponse`](/api-reference/sdk/python/types#asrresponse-objects): `heard.text` is the full transcript and `heard.duration` is the clip length in seconds. Pass `language="en"` to `transcribe()` to skip auto-detection when you already know the input language. For the lowest latency, feed your LLM's token stream straight into [`stream_websocket()`](/api-reference/sdk/python/resources#stream_websocket) instead of waiting for the full reply string — see [Realtime: LLM tokens → speech](/developer-guide/sdk-guide/cookbook/realtime-llm-to-speech). ## Reply in the caller's voice `reference_id` points the reply at a saved voice. Drop it to use the default voice, or clone the caller's voice from the same clip you just transcribed by passing `references` instead — see [Instant voice cloning](/developer-guide/sdk-guide/cookbook/instant-voice-cloning). ## Related * [Speech-to-Text guide](/features/speech-to-text) * [Realtime: LLM tokens → speech](/developer-guide/sdk-guide/cookbook/realtime-llm-to-speech) * [Stream TTS to a file](/developer-guide/sdk-guide/cookbook/streaming-to-file) # Authentication Source: https://docs.fish.audio/developer-guide/sdk-guide/python/authentication Configure API authentication for the Fish Audio Python SDK ## Get Your API Key Sign up for a free Fish Audio account to get started with our API. 1. Go to [fish.audio/auth/signup](https://fish.audio/auth/signup) 2. Fill in your details to create an account, complete steps to verify your account. 3. Log in to your account and navigate to the [API section](https://fish.audio/app/api-keys) Once you have an account, you'll need an API key to authenticate your requests. 1. Log in to your [Fish Audio Dashboard](https://fish.audio/app/api-keys/) 2. Navigate to the API Keys section 3. Click "Create New Key" and give it a descriptive name, set a expiration if desired 4. Copy your key and store it securely Keep your API key secret! Never commit it to version control or share it publicly. ## Client Initialization Initialize the [`FishAudio`](/api-reference/sdk/python/client#fishaudio-objects) client with your API key: The most secure approach is using environment variables: ```python theme={null} from fishaudio import FishAudio # Automatically reads from FISH_API_KEY environment variable client = FishAudio() ``` Set the environment variable in your shell: ```bash theme={null} export FISH_API_KEY=your_api_key_here ``` Or create a `.env` file in your project root: ```bash theme={null} FISH_API_KEY=your_api_key_here ``` Then load it using `python-dotenv`: ```python theme={null} from dotenv import load_dotenv from fishaudio import FishAudio # Load environment variables from .env file load_dotenv() client = FishAudio() ``` Using environment variables keeps your API key out of your codebase and makes it easy to use different keys for development and production. Provide the API key directly when initializing the client: ```python theme={null} from fishaudio import FishAudio client = FishAudio(api_key="your_api_key_here") ``` This approach is less secure. Never commit code containing your actual API key. Use this only for quick testing or when loading the key from a secure secrets manager. If you're using a proxy or custom endpoint: ```python theme={null} from fishaudio import FishAudio client = FishAudio( api_key="your_api_key", base_url="https://your-proxy-domain.com" ) ``` This is useful for: * Corporate proxies * Development/staging environments * Self-hosted deployments ## Verifying Authentication Test your authentication by making a simple API call to check your account credits: ```python focus={7-9} theme={null} from fishaudio import FishAudio from fishaudio.exceptions import AuthenticationError try: client = FishAudio() # Check account credits (requires valid authentication) credits = client.account.get_credits() print(f"Authentication successful! Credits: {credits.credit}") except AuthenticationError: print("Authentication failed. Check your API key.") ``` Handle [`AuthenticationError`](/api-reference/sdk/python/exceptions#authenticationerror-objects) when verifying authentication. The example uses [`get_credits()`](/api-reference/sdk/python/resources#get_credits) to verify the authentication works. ## Next Steps Generate speech with the authenticated client Clone voices and create custom models Check credits and manage your account Handle authentication errors properly # Errors & Retries Source: https://docs.fish.audio/developer-guide/sdk-guide/python/errors Exception types, retry strategy, and timeouts in the Fish Audio Python SDK ## Prerequisites Sign up for a free Fish Audio account to get started with our API. 1. Go to [fish.audio/auth/signup](https://fish.audio/auth/signup) 2. Fill in your details to create an account, complete steps to verify your account. 3. Log in to your account and navigate to the [API section](https://fish.audio/app/api-keys) Once you have an account, you'll need an API key to authenticate your requests. 1. Log in to your [Fish Audio Dashboard](https://fish.audio/app/api-keys/) 2. Navigate to the API Keys section 3. Click "Create New Key" and give it a descriptive name, set a expiration if desired 4. Copy your key and store it securely Keep your API key secret! Never commit it to version control or share it publicly. ## Exception hierarchy Every SDK error inherits from [`FishAudioError`](/api-reference/sdk/python/exceptions#fishaudioerror-objects). HTTP failures raise [`APIError`](/api-reference/sdk/python/exceptions#apierror-objects) or one of its subclasses, which expose `.status`, `.message`, and `.body`. | Exception | Raised when | Notes | | --------------------- | ------------------------------------------------------------ | ---------------------------------------------------------- | | `AuthenticationError` | `401` | Missing or invalid API key | | `PermissionError` | `403` | Key lacks permission for the resource | | `NotFoundError` | `404` | Voice model id not found | | `RateLimitError` | `429` | Rate limit / quota exceeded | | `ServerError` | `5xx` | Transient server-side failure | | `APIError` | any other non-2xx | Base for the above; `status == 422` for invalid parameters | | `WebSocketError` | realtime stream failed mid-session | Reconnect rather than retrying the same socket | | `DependencyError` | a required system tool is missing (e.g. ffmpeg for `play()`) | Carries `.dependency` and `.install_command` | There is no separate `ValidationError` raised at runtime. Invalid request parameters come back as an `APIError` with `status == 422` — catch `APIError`, not `ValidationError`. ## Handling errors ```python Synchronous theme={null} from fishaudio import FishAudio from fishaudio.exceptions import ( AuthenticationError, RateLimitError, NotFoundError, APIError, FishAudioError, ) client = FishAudio() try: audio = client.tts.convert(text="Hello!", reference_id="maybe-missing") except AuthenticationError: print("Invalid API key") except RateLimitError: print("Rate limited — back off and retry") except NotFoundError: print("That voice model does not exist") except APIError as e: print(f"API error {e.status}: {e.message}") # includes 422 validation except FishAudioError as e: print(f"SDK error: {e}") # e.g. WebSocketError, DependencyError ``` ```python Asynchronous theme={null} import asyncio from fishaudio import AsyncFishAudio from fishaudio.exceptions import RateLimitError, APIError, FishAudioError async def main(): async with AsyncFishAudio() as client: try: audio = await client.tts.convert(text="Hello!") except RateLimitError: print("Rate limited — back off and retry") except APIError as e: print(f"API error {e.status}: {e.message}") except FishAudioError as e: print(f"SDK error: {e}") asyncio.run(main()) ``` ## Retries The Python client does **not** retry automatically — each call makes a single request and raises on failure. Add your own backoff where it matters, typically around `RateLimitError` and `ServerError`: ```python theme={null} import time from fishaudio import FishAudio from fishaudio.exceptions import RateLimitError, ServerError client = FishAudio() def convert_with_retry(text: str, max_retries: int = 3) -> bytes: for attempt in range(max_retries): try: return client.tts.convert(text=text) except (RateLimitError, ServerError): if attempt == max_retries - 1: raise time.sleep(2 ** attempt) # exponential backoff raise RuntimeError("unreachable") ``` `RequestOptions` accepts a `max_retries` field, but the current client does not act on it — use an explicit loop like the one above. ## Timeouts The request timeout is set on the client (seconds; default `240`): ```python theme={null} from fishaudio import FishAudio client = FishAudio(timeout=30.0) ``` Override headers or timeout for a single request with `request_options`: ```python theme={null} from fishaudio.core.request_options import RequestOptions audio = client.tts.convert( text="Hello!", request_options=RequestOptions(timeout=15.0, additional_headers={"X-Trace": "abc"}), ) ``` If you inject your own `httpx_client`, the SDK uses it as-is — the client-level `timeout`, `base_url`, and the `Authorization` header are **not** applied to it. Configure those on the client you pass in. ## Related * [Exceptions API reference](/api-reference/sdk/python/exceptions) * [Real-time WebSocket](/features/realtime-streaming) — `WebSocketError` handling # SDK Quickstart Source: https://docs.fish.audio/developer-guide/sdk-guide/quickstart Install a Fish Audio SDK, authenticate, and generate your first audio in under a minute The fastest path from zero to playable audio with the official Fish Audio SDKs. By the end you'll have a script that turns text into an MP3. The Python SDK is the recommended starting point and is fully covered below. The JavaScript SDK is in early release — see the [JavaScript SDK guide](/api-reference/sdk/javascript/api-reference) for its current surface. ## 1. Install ```bash Python theme={null} pip install fish-audio-sdk # optional: local audio playback (needs ffmpeg) pip install "fish-audio-sdk[utils]" ``` ```bash JavaScript theme={null} npm install fish-audio ``` ## 2. Authenticate Sign up for a free Fish Audio account to get started with our API. 1. Go to [fish.audio/auth/signup](https://fish.audio/auth/signup) 2. Fill in your details to create an account, complete steps to verify your account. 3. Log in to your account and navigate to the [API section](https://fish.audio/app/api-keys) Once you have an account, you'll need an API key to authenticate your requests. 1. Log in to your [Fish Audio Dashboard](https://fish.audio/app/api-keys/) 2. Navigate to the API Keys section 3. Click "Create New Key" and give it a descriptive name, set a expiration if desired 4. Copy your key and store it securely Keep your API key secret! Never commit it to version control or share it publicly. Both SDKs read your key from the `FISH_API_KEY` environment variable: ```bash theme={null} export FISH_API_KEY=your_api_key_here ``` ## 3. Generate your first audio ```python Python theme={null} from fishaudio import FishAudio from fishaudio.utils import save client = FishAudio() # reads FISH_API_KEY audio = client.tts.convert(text="Hello from Fish Audio!") save(audio, "output.mp3") ``` ```typescript JavaScript theme={null} import { FishAudioClient, play } from "fish-audio"; const client = new FishAudioClient(); // reads FISH_API_KEY const audio = await client.textToSpeech.convert({ text: "Hello from Fish Audio!" }); await play(audio); ``` Run it, and you'll have `output.mp3` (Python) or local playback (JavaScript). That's it — you're generating speech. Want async in Python? Every method mirrors onto `AsyncFishAudio`: `async with AsyncFishAudio() as client: audio = await client.tts.convert(text="...")`. ## Next steps Voices, formats, prosody, and model selection Instant cloning and persistent voice models Stream LLM tokens to speech as they arrive Exception types, retries, and timeouts Task-focused recipes Full Python SDK reference # Docker Deployment Source: https://docs.fish.audio/developer-guide/self-hosting/docker-deployment Deploy Fish Audio models using Docker containers Fish Audio provides Docker images for both WebUI and API server deployments. You can use pre-built images from Docker Hub or build custom images locally. ## Prerequisites Before deploying with Docker, ensure you have: * **Docker** and **Docker Compose** installed * **NVIDIA Docker runtime** (for GPU support) * At least **12GB GPU memory** for CUDA inference * Downloaded model weights (see [Running Inference](/developer-guide/self-hosting/running-inference#download-weights)) ## Pre-built Images Fish Audio provides ready-to-use Docker images on Docker Hub: | Image | Description | Best For | | ------------------------------------------ | ----------------------- | -------------------------------- | | `fishaudio/fish-speech:latest-webui-cuda` | WebUI with CUDA support | Interactive development with GPU | | `fishaudio/fish-speech:latest-webui-cpu` | WebUI CPU-only | Testing without GPU | | `fishaudio/fish-speech:latest-server-cuda` | API server with CUDA | Production deployments with GPU | | `fishaudio/fish-speech:latest-server-cpu` | API server CPU-only | Low-traffic CPU deployments | For production use, we recommend using specific version tags instead of `latest` to ensure consistency across deployments. ## Quick Start with Docker Run The fastest way to get started is using `docker run`: ### WebUI Deployment ```bash theme={null} # Create directories for model weights and reference audio mkdir -p checkpoints references # Start WebUI with CUDA support (recommended) docker run -d \ --name fish-speech-webui \ --gpus all \ -p 7860:7860 \ -v ./checkpoints:/app/checkpoints \ -v ./references:/app/references \ -e COMPILE=1 \ fishaudio/fish-speech:latest-webui-cuda # For CPU-only deployment docker run -d \ --name fish-speech-webui-cpu \ -p 7860:7860 \ -v ./checkpoints:/app/checkpoints \ -v ./references:/app/references \ fishaudio/fish-speech:latest-webui-cpu ``` Access the WebUI at `http://localhost:7860` ### API Server Deployment ```bash theme={null} # Start API server with CUDA support docker run -d \ --name fish-speech-server \ --gpus all \ -p 8080:8080 \ -v ./checkpoints:/app/checkpoints \ -v ./references:/app/references \ -e COMPILE=1 \ fishaudio/fish-speech:latest-server-cuda # For CPU-only deployment docker run -d \ --name fish-speech-server-cpu \ -p 8080:8080 \ -v ./checkpoints:/app/checkpoints \ -v ./references:/app/references \ fishaudio/fish-speech:latest-server-cpu ``` Access the API documentation at `http://localhost:8080` Enable the `COMPILE=1` environment variable for \~10x faster inference on CUDA deployments. This uses `torch.compile` to optimize the model. ## Docker Compose Deployment For development or customization, Docker Compose provides easier configuration management: ### Setup ```bash theme={null} # Clone the repository git clone https://github.com/fishaudio/fish-speech.git cd fish-speech ``` ### Start Services ```bash theme={null} # Start WebUI with CUDA docker compose --profile webui up # Start WebUI with compile optimization COMPILE=1 docker compose --profile webui up # Start API server docker compose --profile server up # Start API server with compile optimization COMPILE=1 docker compose --profile server up # For CPU-only deployment BACKEND=cpu docker compose --profile webui up ``` Run containers in detached mode by adding the `-d` flag: `docker compose --profile webui up -d` ### Environment Variables Customize deployment using environment variables or a `.env` file: ```bash theme={null} # .env file example BACKEND=cuda # or cpu COMPILE=1 # Enable compile optimization GRADIO_PORT=7860 # WebUI port API_PORT=8080 # API server port UV_VERSION=0.8.15 # UV package manager version ``` ## Manual Docker Build For advanced users who need custom configurations: ### Build WebUI Image ```bash theme={null} # Build with CUDA support docker build \ --platform linux/amd64 \ -f docker/Dockerfile \ --build-arg BACKEND=cuda \ --build-arg CUDA_VER=12.6.0 \ --build-arg UV_EXTRA=cu126 \ --target webui \ -t fish-speech-webui:cuda . # Build CPU-only (supports multi-platform) docker build \ --platform linux/amd64,linux/arm64 \ -f docker/Dockerfile \ --build-arg BACKEND=cpu \ --target webui \ -t fish-speech-webui:cpu . ``` ### Build API Server Image ```bash theme={null} # Build with CUDA support docker build \ --platform linux/amd64 \ -f docker/Dockerfile \ --build-arg BACKEND=cuda \ --build-arg CUDA_VER=12.6.0 \ --build-arg UV_EXTRA=cu126 \ --target server \ -t fish-speech-server:cuda . ``` ### Build Development Image ```bash theme={null} # Build development image with all tools docker build \ --platform linux/amd64 \ -f docker/Dockerfile \ --build-arg BACKEND=cuda \ --target dev \ -t fish-speech-dev:cuda . ``` ### Build Arguments | Argument | Options | Default | Description | | ------------ | ------------------------- | -------- | ------------------- | | `BACKEND` | `cuda`, `cpu` | `cuda` | Compute backend | | `CUDA_VER` | `12.6.0`, etc. | `12.6.0` | CUDA version | | `UV_EXTRA` | `cu126`, `cu128`, `cu129` | `cu126` | UV extra for CUDA | | `UBUNTU_VER` | `24.04`, etc. | `24.04` | Ubuntu base version | | `PY_VER` | `3.12`, etc. | `3.12` | Python version | ## Volume Mounts Both Docker run and Compose methods require these volume mounts: | Host Path | Container Path | Purpose | | --------------- | ------------------ | --------------------------------------- | | `./checkpoints` | `/app/checkpoints` | Model weights directory | | `./references` | `/app/references` | Reference audio files for voice cloning | Ensure model weights are downloaded and placed in the `./checkpoints` directory before starting containers. See [Running Inference](/developer-guide/self-hosting/running-inference#download-weights) for download instructions. ## Environment Variables Reference ### WebUI Configuration | Variable | Default | Description | | -------------------- | --------- | ---------------------------- | | `GRADIO_SERVER_NAME` | `0.0.0.0` | WebUI server host | | `GRADIO_SERVER_PORT` | `7860` | WebUI server port | | `GRADIO_SHARE` | `false` | Enable Gradio public sharing | ### API Server Configuration | Variable | Default | Description | | ----------------- | --------- | --------------- | | `API_SERVER_NAME` | `0.0.0.0` | API server host | | `API_SERVER_PORT` | `8080` | API server port | ### Model Configuration | Variable | Default | Description | | ------------------------- | ----------------------------------------- | -------------------------- | | `LLAMA_CHECKPOINT_PATH` | `checkpoints/openaudio-s1-mini` | Path to model weights | | `DECODER_CHECKPOINT_PATH` | `checkpoints/openaudio-s1-mini/codec.pth` | Path to decoder weights | | `DECODER_CONFIG_NAME` | `modded_dac_vq` | Decoder configuration name | ### Performance Optimization | Variable | Default | Description | | --------- | ------- | -------------------------------------------------- | | `COMPILE` | `0` | Enable torch.compile for \~10x speedup (CUDA only) | ## Container Management ### View Logs ```bash theme={null} # Docker run docker logs fish-speech-webui # Docker Compose docker compose logs webui ``` ### Stop Containers ```bash theme={null} # Docker run docker stop fish-speech-webui # Docker Compose docker compose down ``` ### Update Images ```bash theme={null} # Pull latest images docker pull fishaudio/fish-speech:latest-webui-cuda # Restart containers with new image docker compose --profile webui up -d ``` ## GPU Support ### Prerequisites Install NVIDIA Container Toolkit: ```bash theme={null} # Ubuntu/Debian distribution=$(. /etc/os-release;echo $ID$VERSION_ID) curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | \ sudo tee /etc/apt/sources.list.d/nvidia-docker.list sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit sudo systemctl restart docker ``` ### Verify GPU Access ```bash theme={null} docker run --rm --gpus all nvidia/cuda:12.6.0-base-ubuntu24.04 nvidia-smi ``` GPU support requires NVIDIA Docker runtime. For CPU-only deployment, remove the `--gpus all` flag and use CPU images. ## Troubleshooting ### Container Won't Start Check logs for errors: ```bash theme={null} docker logs fish-speech-webui ``` Common issues: * Missing model weights in `./checkpoints` * Port already in use (change port mapping) * Insufficient GPU memory ### GPU Not Detected Verify NVIDIA Docker runtime is installed: ```bash theme={null} docker run --rm --gpus all nvidia/cuda:12.6.0-base-ubuntu24.04 nvidia-smi ``` ### Performance Issues 1. Enable compile optimization: `COMPILE=1` 2. Ensure GPU is being used (check with `nvidia-smi`) 3. Verify sufficient GPU memory is available ## Next Steps * **[Run inference](/developer-guide/self-hosting/running-inference)** - Learn how to generate speech * **[Download models](https://huggingface.co/fishaudio)** - Get pre-trained weights * **[API documentation](/api-reference/introduction)** - Integrate with your applications # Local Model Setup Source: https://docs.fish.audio/developer-guide/self-hosting/local-setup Install and configure Fish Audio models for local inference This guide is for advanced users who want to self-host Fish Audio models. For most users, we recommend using the [Fish Audio API](https://fish.audio) for easier integration and automatic updates. ## Prerequisites Before you begin, ensure you have: * **GPU**: 12GB VRAM minimum (for inference) * **OS**: Linux or WSL (Windows Subsystem for Linux) * **System dependencies**: Audio processing libraries Install required system packages: ```bash theme={null} apt install portaudio19-dev libsox-dev ffmpeg ``` ## Installation Methods Fish Audio supports multiple installation methods. Choose the one that best fits your development environment. ### Conda Installation Conda provides a stable, isolated Python environment: ```bash theme={null} # Create a new environment with Python 3.12 conda create -n fish-speech python=3.12 conda activate fish-speech # GPU installation (choose your CUDA version: cu126, cu128, cu129) pip install -e .[cu129] # CPU-only installation (slower, not recommended for production) pip install -e .[cpu] # Default installation (uses PyTorch default index) pip install -e . ``` For best performance, match your CUDA version with your GPU driver. Use `nvidia-smi` to check your CUDA version. ### UV Installation [UV](https://github.com/astral-sh/uv) provides faster dependency resolution and installation: ```bash theme={null} # GPU installation (choose your CUDA version: cu126, cu128, cu129) uv sync --python 3.12 --extra cu129 # CPU-only installation uv sync --python 3.12 --extra cpu ``` UV is recommended for faster setup times, especially when working with large dependency trees. ### Intel Arc XPU Support For Intel Arc GPU users, install with XPU support: ```bash theme={null} # Create environment conda create -n fish-speech python=3.12 conda activate fish-speech # Install required C++ standard library conda install libstdcxx -c conda-forge # Install PyTorch with Intel XPU support pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/xpu # Install Fish Speech pip install -e . ``` The `--compile` optimization flag is not supported on Windows and macOS. To use compile acceleration, you need to install Triton manually. ## Repository Setup Clone the Fish Speech repository to get started: ```bash theme={null} git clone https://github.com/fishaudio/fish-speech.git cd fish-speech ``` Then follow one of the installation methods above. ## Next Steps Once installation is complete, you can: * **[Set up Docker deployment](/developer-guide/self-hosting/docker-deployment)** - Use containerized deployment for easier management * **[Run inference](/developer-guide/self-hosting/running-inference)** - Start generating speech with your local models * **Download models** - Get pre-trained weights from [Hugging Face](https://huggingface.co/fishaudio) ## Hardware Recommendations For optimal performance: | Use Case | Recommended GPU | VRAM | Expected Speed | | ----------- | --------------- | ----- | ----------------------- | | Development | RTX 3060 | 12GB | \~1:15 real-time factor | | Production | RTX 4090 | 24GB | \~1:7 real-time factor | | Enterprise | A100 | 40GB+ | \~1:5 real-time factor | Real-time factor indicates how much faster than real-time the model can generate audio. For example, 1:7 means generating 1 minute of audio takes \~8.5 seconds. ## Troubleshooting ### CUDA Out of Memory If you encounter CUDA out of memory errors: 1. Reduce batch size in inference settings 2. Use `--half` flag for FP16 inference 3. Close other GPU-intensive applications ### Package Installation Errors If you encounter dependency conflicts: 1. Try using UV instead of pip for better dependency resolution 2. Create a fresh conda environment 3. Ensure you're using Python 3.12 (other versions may have compatibility issues) ## Community Support Need help with local setup? * Join our [Discord community](https://discord.gg/dF9Db2Tt3Y) for community support * Check [GitHub Issues](https://github.com/fishaudio/fish-speech/issues) for known problems * Contact [enterprise support](mailto:support@fish.audio) for commercial deployments # Running Inference Source: https://docs.fish.audio/developer-guide/self-hosting/running-inference Generate speech using self-hosted Fish Audio models Fish Audio supports multiple inference methods: command line, HTTP API, WebUI, and GUI. Choose the method that best fits your workflow. This guide assumes you have already [installed Fish Audio locally](/developer-guide/self-hosting/local-setup) or [set up Docker deployment](/developer-guide/self-hosting/docker-deployment). ## Download Weights Before running inference, download the required model weights from Hugging Face: ```bash theme={null} # Install Hugging Face CLI (if not already installed) pip install huggingface_hub[cli] # or uv tool install huggingface_hub[cli] # Download Fish Audio S1-mini weights hf download fishaudio/openaudio-s1-mini --local-dir checkpoints/openaudio-s1-mini ``` **Fish Audio S1-mini** is the open-source distilled version (0.5B parameters) optimized for local deployment. The full **S1** model (4B parameters) is available exclusively on [Fish Audio cloud](https://fish.audio). ## Command Line Inference Command line inference provides maximum control and is ideal for scripting and batch processing. ### Step 1: Extract VQ Tokens from Reference Audio First, encode your reference audio to get voice characteristics: ```bash theme={null} python fish_speech/models/dac/inference.py \ -i "reference_audio.wav" \ --checkpoint-path "checkpoints/openaudio-s1-mini/codec.pth" ``` This generates two files: * `fake.npy` - VQ tokens representing voice characteristics * `fake.wav` - Reconstructed audio for verification **Skip this step if you want random voice generation** - the model can generate speech without reference audio. ### Step 2: Generate Semantic Tokens from Text Convert your text to semantic tokens using the language model: ```bash theme={null} python fish_speech/models/text2semantic/inference.py \ --text "The text you want to convert to speech" \ --prompt-text "Transcription of your reference audio" \ --prompt-tokens "fake.npy" \ --compile ``` **Parameters:** * `--text`: The text to synthesize * `--prompt-text`: Transcription of the reference audio (for voice cloning) * `--prompt-tokens`: Path to VQ tokens from Step 1 (for voice cloning) * `--compile`: Enable kernel fusion for faster inference (\~10x speedup on RTX 4090) For random voice generation, omit `--prompt-text` and `--prompt-tokens` parameters. This creates a file named `codes_N.npy` (where N starts from 0) containing semantic tokens. For GPUs that don't support bf16 (bfloat16), add the `--half` flag to use fp16 instead. ### Step 3: Generate Audio from Semantic Tokens Finally, convert semantic tokens to audio: ```bash theme={null} python fish_speech/models/dac/inference.py \ -i "codes_0.npy" ``` This generates the final audio file. ### Full Example Here's a complete workflow for voice cloning: ```bash theme={null} # 1. Encode reference audio python fish_speech/models/dac/inference.py \ -i "my_voice.wav" \ --checkpoint-path "checkpoints/openaudio-s1-mini/codec.pth" # 2. Generate semantic tokens python fish_speech/models/text2semantic/inference.py \ --text "Hello, this is a test of voice cloning." \ --prompt-text "This is my reference voice recording." \ --prompt-tokens "fake.npy" \ --compile # 3. Generate final audio python fish_speech/models/dac/inference.py \ -i "codes_0.npy" ``` ## HTTP API Inference The HTTP API provides a programmatic interface for integrations and production deployments. ### Start API Server ```bash theme={null} # With local installation python -m tools.api_server \ --listen 0.0.0.0:8080 \ --llama-checkpoint-path "checkpoints/openaudio-s1-mini" \ --decoder-checkpoint-path "checkpoints/openaudio-s1-mini/codec.pth" \ --decoder-config-name modded_dac_vq # With UV uv run tools/api_server.py \ --listen 0.0.0.0:8080 \ --llama-checkpoint-path "checkpoints/openaudio-s1-mini" \ --decoder-checkpoint-path "checkpoints/openaudio-s1-mini/codec.pth" \ --decoder-config-name modded_dac_vq ``` Add the `--compile` flag to enable torch.compile optimization for faster inference. ### Access API Documentation Once the server is running, access the interactive API documentation at: ``` http://localhost:8080/docs ``` The API provides endpoints for: * Text-to-speech synthesis * Voice cloning with reference audio * Batch processing * Model information ### Example API Request ```bash theme={null} curl -X POST "http://localhost:8080/v1/tts" \ -H "Content-Type: application/json" \ -d '{ "text": "Hello, this is a test", "reference_audio": "base64_encoded_audio", "reference_text": "Reference transcription" }' ``` ## WebUI Inference The WebUI provides an intuitive interface for interactive testing and development. ### Start WebUI ```bash theme={null} # With all parameters python -m tools.run_webui \ --llama-checkpoint-path "checkpoints/openaudio-s1-mini" \ --decoder-checkpoint-path "checkpoints/openaudio-s1-mini/codec.pth" \ --decoder-config-name modded_dac_vq # Or use defaults (auto-detects models in checkpoints/) python -m tools.run_webui ``` Add the `--compile` flag for faster inference during interactive sessions. ### Access WebUI The WebUI starts on port 7860 by default. Access it at: ``` http://localhost:7860 ``` ### Configure with Environment Variables Customize the WebUI using Gradio environment variables: ```bash theme={null} # Enable public sharing GRADIO_SHARE=1 python -m tools.run_webui # Change server port GRADIO_SERVER_PORT=8080 python -m tools.run_webui # Change server name GRADIO_SERVER_NAME=0.0.0.0 python -m tools.run_webui ``` ### Using Reference Audio Library For faster workflow, pre-save reference audio: 1. Create a `references/` directory in the project root 2. Create subdirectories named by voice ID: `references//` 3. Place files in each subdirectory: * `sample.wav` - Reference audio file * `sample.lab` - Text transcription of the audio Example structure: ``` references/ ├── alice/ │ ├── sample.wav │ └── sample.lab └── bob/ ├── sample.wav └── sample.lab ``` These references will appear as selectable options in the WebUI. ## GUI Inference For users who prefer a native desktop application, a PyQt6-based GUI is available. ### Download GUI Client Download the latest release from the [Fish Speech GUI repository](https://github.com/AnyaCoder/fish-speech-gui/releases). **Supported platforms:** * Linux * Windows * macOS ### Connect to API Server The GUI client connects to a running API server (see [HTTP API Inference](#http-api-inference) above). 1. Start the API server 2. Launch the GUI client 3. Configure the API endpoint (default: `http://localhost:8080`) ## Docker Inference If you're using Docker deployment, refer to the [Docker Deployment guide](/developer-guide/self-hosting/docker-deployment) for detailed instructions on: * Running pre-built WebUI containers * Running pre-built API server containers * Customizing container configuration * Volume mounts for models and references Quick example: ```bash theme={null} # Start WebUI with Docker docker run -d \ --name fish-speech-webui \ --gpus all \ -p 7860:7860 \ -v ./checkpoints:/app/checkpoints \ -v ./references:/app/references \ -e COMPILE=1 \ fishaudio/fish-speech:latest-webui-cuda ``` ## Performance Optimization ### Enable Compilation Torch compilation provides \~10x speedup on compatible GPUs: ```bash theme={null} # Add --compile flag to any inference command python -m tools.api_server --compile ... ``` Compilation requires: * CUDA-compatible GPU * Triton library (not supported on Windows/macOS) * First run will be slow due to compilation overhead ### Use Mixed Precision For GPUs without bf16 support, use fp16: ```bash theme={null} python fish_speech/models/text2semantic/inference.py --half ... ``` ### Batch Processing For multiple audio generations, use batch processing to amortize model loading overhead: ```python theme={null} # Example batch processing script import fish_speech model = fish_speech.load_model("checkpoints/openaudio-s1-mini") texts = ["First sentence", "Second sentence", "Third sentence"] for text in texts: audio = model.synthesize(text) audio.save(f"output_{texts.index(text)}.wav") ``` ## Emotion Control Fish Audio S1 supports emotional markers for expressive speech synthesis: ### Basic Emotions ``` (angry) (sad) (excited) (surprised) (satisfied) (delighted) (scared) (worried) (upset) (nervous) (frustrated) (depressed) (empathetic) (embarrassed) (disgusted) (moved) (proud) (relaxed) (grateful) (confident) (interested) (curious) (confused) (joyful) ``` ### Advanced Emotions ``` (disdainful) (unhappy) (anxious) (hysterical) (indifferent) (impatient) (guilty) (scornful) (panicked) (furious) (reluctant) (keen) (disapproving) (negative) (denying) (astonished) (serious) (sarcastic) (conciliative) (comforting) (sincere) (sneering) (hesitating) (yielding) (painful) (awkward) (amused) ``` ### Tone Markers ``` (in a hurry tone) (shouting) (screaming) (whispering) (soft tone) ``` ### Special Effects ``` (laughing) (chuckling) (sobbing) (crying loudly) (sighing) (panting) (groaning) (crowd laughing) (background laughter) (audience laughing) ``` ### Example Usage ```bash theme={null} python fish_speech/models/text2semantic/inference.py \ --text "(excited)This is amazing! (laughing)Ha ha ha!" \ --compile ``` Emotion control is currently supported for English, Chinese, and Japanese. More languages coming soon! For more details, see the [Emotion Control guide](/developer-guide/core-features/emotions). ## Troubleshooting ### Out of Memory Errors If you encounter CUDA out of memory errors: 1. Reduce input text length 2. Use `--half` flag for fp16 inference 3. Close other GPU applications 4. Use a smaller batch size ### Slow Inference To improve speed: 1. Enable `--compile` flag 2. Verify GPU is being used (check with `nvidia-smi`) 3. Ensure CUDA version matches PyTorch installation 4. Use fp16 instead of bf16 on older GPUs ### Poor Audio Quality For better quality: 1. Use high-quality reference audio (clear, no background noise) 2. Ensure reference text accurately matches reference audio 3. Use 10-30 seconds of reference audio 4. See [Voice Cloning Best Practices](/developer-guide/best-practices/voice-cloning) ### Model Loading Errors If models fail to load: 1. Verify model weights are downloaded completely 2. Check checkpoint paths are correct 3. Ensure sufficient disk space 4. Re-download weights if corrupted ## Next Steps * **[Emotion Control Best Practices](/developer-guide/best-practices/emotion-control)** - Master expressive speech * **[Voice Cloning Best Practices](/developer-guide/best-practices/voice-cloning)** - Optimize voice cloning quality * **[API Reference](/api-reference/introduction)** - Integrate with your applications * **[Cloud API](https://fish.audio)** - Compare with managed service performance # Manage Voices Source: https://docs.fish.audio/features/manage-voices List, inspect, update, and delete your voice models Every voice you [clone](/features/voice-cloning) becomes a model you own. List your library, look up a model's details, rename or re-share it, and delete what you no longer need — all from the API directly, the Python library, or JavaScript. No code — manage voices in the browser. Every endpoint for voice models. Search and reuse Library voices. ## When to use it List the voices you've created or saved. Look up the `reference_id` to use in [Text to Speech](/features/text-to-speech). Rename, re-tag, or change a model's visibility. Delete models you no longer use. ## List your voices Page through your library. The response carries the `total` count and the `items` for the current page. ```python Python theme={null} from fishaudio import FishAudio client = FishAudio() # reads FISH_API_KEY page = client.voices.list(self_only=True, page_size=20) print(f"{page.total} voices") for v in page.items: print(v.id, v.title, v.state, v.visibility) ``` ```bash API (curl) theme={null} curl "https://api.fish.audio/model?self=true&page_size=20" \ --header "Authorization: Bearer $FISH_API_KEY" # Response: { "total": 42, "items": [ ... ], "has_more": true } ``` ```javascript JavaScript theme={null} import { FishAudioClient } from "fish-audio"; const client = new FishAudioClient({ apiKey: process.env.FISH_API_KEY }); const page = await client.voices.search({ page_size: 20 }); console.log(`${page.total} voices`); for (const v of page.items) { console.log(v._id ?? v.id, v.title, v.state, v.visibility); } ``` ## Get, update, and delete Use a voice **id** to inspect a single model, change its metadata, or remove it. ```python Python theme={null} # Inspect one model voice = client.voices.get("YOUR_VOICE_ID") print(voice.title, voice.state) # Update metadata (only the fields you pass change) client.voices.update( "YOUR_VOICE_ID", title="Updated title", visibility="unlist", ) # Delete client.voices.delete("YOUR_VOICE_ID") ``` ```bash API (curl) theme={null} # Inspect one model curl https://api.fish.audio/model/YOUR_VOICE_ID \ --header "Authorization: Bearer $FISH_API_KEY" # Update metadata curl --request PATCH https://api.fish.audio/model/YOUR_VOICE_ID \ --header "Authorization: Bearer $FISH_API_KEY" \ --header "Content-Type: application/json" \ --data '{ "title": "Updated title", "visibility": "unlist" }' # Delete curl --request DELETE https://api.fish.audio/model/YOUR_VOICE_ID \ --header "Authorization: Bearer $FISH_API_KEY" ``` ## Implementation details ### Filtering and pagination Narrow the list with `title`, `tags`, or `language`, and page with `page_size` and `page_number`. Omit `self_only` (API: `self`) to search the public [Voice Library](/overview/platform) instead of just your own models. ```python Python theme={null} page = client.voices.list( self_only=True, title="narration", page_size=50, page_number=2, ) ``` ```bash API (curl) theme={null} curl "https://api.fish.audio/model?self=true&title=narration&page_size=50&page_number=2" \ --header "Authorization: Bearer $FISH_API_KEY" ``` ### Visibility Switch a model between `private`, `unlist` (shareable link), and `public` (listed in the Voice Library) with `update`. Publishing a model lets anyone use it as a `reference_id`. ## Going further Create a new model from audio samples. Use any voice id as `reference_id`. Every endpoint for listing and managing models. The full `voices` resource surface. # Realtime Streaming Source: https://docs.fish.audio/features/realtime-streaming Stream audio as it generates for the lowest latency Start playing audio before the whole clip is ready. Fish Audio streams speech in chunks, so your users hear the first words in a fraction of a second — essential for voice agents and live narration. Two modes: **HTTP streaming** for text you already have, and **WebSocket** for text that arrives incrementally (like LLM tokens). The live TTS WebSocket protocol. LLM-to-speech and voice agents. Tuning latency for production. ## When to use it Conversational AI where time-to-first-audio matters. Speak tokens as your model produces them — no waiting for the full reply. Long-form content that should start playing immediately. Anywhere a few hundred milliseconds of latency is noticeable. ## Stream text you already have When you have the full string, stream the audio chunks as they generate and write or play them immediately. ```python Python theme={null} from fishaudio import FishAudio client = FishAudio() # reads FISH_API_KEY with open("out.mp3", "wb") as f: for chunk in client.tts.stream(text="Streaming keeps latency low."): f.write(chunk) # or send to a speaker / socket as it arrives # Or collect the whole stream into one bytes object: audio = client.tts.stream(text="Streaming keeps latency low.").collect() ``` ```bash API (curl) theme={null} curl --request POST https://api.fish.audio/v1/tts \ --header "Authorization: Bearer $FISH_API_KEY" \ --header "Content-Type: application/json" \ --header "model: s2-pro" \ --no-buffer \ --data '{ "text": "Streaming keeps latency low.", "format": "mp3" }' \ --output out.mp3 ``` ```javascript JavaScript theme={null} import { FishAudioClient } from "fish-audio"; import { createWriteStream } from "fs"; const client = new FishAudioClient({ apiKey: process.env.FISH_API_KEY }); // convert() returns a ReadableStream — write each chunk the // moment it arrives instead of waiting for the whole clip. const stream = await client.textToSpeech.convert( { text: "Streaming keeps latency low.", format: "mp3" }, "s2-pro" ); const file = createWriteStream("out.mp3"); for await (const chunk of stream) { file.write(Buffer.from(chunk)); // or forward to a speaker / socket as it arrives } file.end(); ``` `--no-buffer` tells curl to write each chunk as it arrives instead of waiting for the full response. ## Stream from an LLM When text arrives token by token, feed a generator to `stream_websocket`. It opens a WebSocket, sends text as you produce it, and yields audio chunks back — so speech keeps pace with your model. ```python Python theme={null} from fishaudio import FishAudio from fishaudio.utils import play client = FishAudio() def llm_tokens(): # Replace with your real streaming LLM call for token in ["The ", "first ", "move ", "sets ", "everything ", "in ", "motion."]: yield token for chunk in client.tts.stream_websocket(llm_tokens(), reference_id="YOUR_VOICE_ID"): play(chunk) # play each chunk the moment it arrives ``` ```bash API (WebSocket) theme={null} # Token-level streaming uses the WebSocket endpoint, not curl. # The Python SDK's stream_websocket() handles the protocol for you. # To build it directly, see the WebSocket reference: # /api-reference/endpoint/websocket/tts-live ``` ## Implementation details ### Which mode to use * **HTTP streaming (`tts.stream`)** — you have the full text up front and want low time-to-first-audio. Simplest option. * **WebSocket (`tts.stream_websocket`)** — text is still being produced (LLM output, live captions). Lets you start speaking before the sentence is finished. ### Lower the latency further * Use a streaming-friendly format like `mp3` or `pcm`. * Keep the connection warm for back-to-back generations. * Pair with a cloned voice via `reference_id` — see [Voice Cloning](/features/voice-cloning). ## Control where audio generates The WebSocket buffers incoming text and generates audio once it has enough context for natural-sounding speech, so you don't need to batch tokens yourself. When you *do* want a clean break — end of a sentence, a deliberate pause, or the end of a turn — yield a `FlushEvent` to force generation immediately. Wrap text in a `TextEvent` if you prefer explicit events over bare strings. ```python theme={null} from fishaudio import FishAudio from fishaudio.types import TextEvent, FlushEvent client = FishAudio() def script(): yield TextEvent(text="First sentence. ") yield "Second sentence. " yield FlushEvent() # generate everything buffered so far, now yield "Third sentence." for chunk in client.tts.stream_websocket(script(), reference_id="YOUR_VOICE_ID"): ... # play or forward each chunk ``` ## Tune latency vs. quality Both streaming paths take a `latency` mode: * `latency="balanced"` (default) — lowest time-to-first-audio. Use it for voice agents and live LLM output. * `latency="normal"` — slightly higher latency, best audio quality. Use it for narration where you can afford a beat. ```python theme={null} for chunk in client.tts.stream_websocket(llm_tokens(), latency="balanced"): ... ``` For finer control, pass a `TTSConfig` with chunk tuning. Smaller chunks emit audio sooner (lower latency); larger chunks give the model more context (smoother prosody): ```python theme={null} from fishaudio.types import TTSConfig config = TTSConfig( latency="balanced", chunk_length=200, # target tokens per generated chunk min_chunk_length=100, # don't emit a chunk shorter than this ) for chunk in client.tts.stream(text="...", config=config): ... ``` ## Stream asynchronously For asyncio apps, `AsyncFishAudio` exposes the same streaming methods. `stream_websocket` accepts an async generator, so you can pipe an async LLM client straight into speech. ```python theme={null} import asyncio from fishaudio import AsyncFishAudio async def main(): client = AsyncFishAudio() async def llm_tokens(): async for token in your_async_llm(): yield token # stream_websocket is an async generator — iterate it, don't await the call async for chunk in client.tts.stream_websocket( llm_tokens(), reference_id="YOUR_VOICE_ID", latency="balanced" ): ... # play or forward each chunk asyncio.run(main()) ``` ## Direct API (no SDK) Token-level streaming runs over the WebSocket endpoint — the SDK's `stream_websocket()` handles framing for you. To speak the protocol directly, send MessagePack frames over the socket; the same `application/msgpack` payload format also works for one-shot HTTP streaming, which is faster to serialize than JSON for large reference audio: ```python theme={null} import os import httpx import ormsgpack payload = {"text": "Streaming keeps latency low.", "format": "mp3", "latency": "balanced"} with httpx.stream( "POST", "https://api.fish.audio/v1/tts", headers={ "Authorization": f"Bearer {os.environ['FISH_API_KEY']}", "Content-Type": "application/msgpack", "model": "s2-pro", }, content=ormsgpack.packb(payload), ) as r: for chunk in r.iter_bytes(): ... # write each chunk as it arrives ``` For the full WebSocket frame sequence, see the [live TTS protocol reference](/api-reference/endpoint/websocket/tts-live). ## Going further Voices, formats, and prosody for every generation. The live TTS protocol, message by message. Tuning latency for production voice apps. `tts.stream` and `tts.stream_websocket`. # Speech to Text Source: https://docs.fish.audio/features/speech-to-text Transcribe audio to text with per-segment timestamps Turn spoken audio into accurate text — with timed segments — using Fish Audio's ASR model. Send an audio file, get back the transcript, its duration, and timestamped segments. Works the same from the API directly, the Python library, or JavaScript. No code — upload audio, get a transcript. Every parameter for `POST /v1/asr`. Captions, batch transcription, and more. ## When to use it Timed segments map straight to SRT/VTT cues. Transcribe recordings for summaries and search. Turn short utterances into text your app can act on. Make audio and video content readable. ## Quick start Read an audio file, send the bytes, get the transcript. Choose your implementation: ```python Python theme={null} from fishaudio import FishAudio client = FishAudio() # reads FISH_API_KEY with open("speech.wav", "rb") as f: result = client.asr.transcribe(audio=f.read(), language="en") print(result.text) ``` ```bash API (curl) theme={null} curl --request POST https://api.fish.audio/v1/asr \ --header "Authorization: Bearer $FISH_API_KEY" \ --form audio=@speech.wav \ --form language=en ``` ```javascript JavaScript theme={null} import { FishAudioClient } from "fish-audio"; import { readFile } from "fs/promises"; const client = new FishAudioClient({ apiKey: process.env.FISH_API_KEY }); const result = await client.speechToText.convert({ audio: new File([await readFile("speech.wav")], "speech.wav"), language: "en", }); console.log(result.text); ``` The response gives you the full `text`, the audio `duration` in seconds, and timed `segments`. ## Read the timestamps Each segment carries `start` and `end` times in seconds — ideal for captions. With the API, ask for them explicitly with `ignore_timestamps=false`. ```python Python theme={null} result = client.asr.transcribe(audio=audio_bytes, language="en", include_timestamps=True) print(f"{result.duration:.1f}s total") for seg in result.segments: print(f"[{seg.start:6.2f} - {seg.end:6.2f}] {seg.text}") ``` ```bash API (curl) theme={null} curl --request POST https://api.fish.audio/v1/asr \ --header "Authorization: Bearer $FISH_API_KEY" \ --form audio=@speech.wav \ --form language=en \ --form ignore_timestamps=false | jq '.segments' # Each segment: { "text": "One", "start": 0.0, "end": 0.24 } ``` In the Python SDK, segment timestamps are **on by default** — pass `include_timestamps=False` to skip them. That's the *inverse* of the API/JavaScript flag `ignore_timestamps`. ## Implementation details ### Language `language` is optional — Fish Audio auto-detects it when you omit it. Pass an ISO code (`en`, `zh`, `ja`, …) to pin it and improve accuracy on short or noisy clips. ```python Python theme={null} # Auto-detect result = client.asr.transcribe(audio=audio_bytes) # Pin the language result = client.asr.transcribe(audio=audio_bytes, language="zh") ``` ```bash API (curl) theme={null} # Omit the form field to auto-detect, or set it explicitly: curl --request POST https://api.fish.audio/v1/asr \ --header "Authorization: Bearer $FISH_API_KEY" \ --form audio=@speech.wav \ --form language=zh ``` ### Input audio Common formats work directly — `wav`, `mp3`, `opus`, and more. Send the raw file bytes; no pre-processing required. The endpoint accepts `multipart/form-data` (shown above) or `application/msgpack`. ### File limits One request transcribes one audio file. The endpoint accepts files up to **20 MB** and **60 minutes** long, with a minimum of **1 second** of audio. For longer recordings, split them into chunks and transcribe each, then stitch the segment timestamps back together (offset each chunk's `start`/`end` by where it began in the full recording). ### Async transcription The Python SDK ships an async client with the same surface — useful when you're transcribing many files concurrently or already running inside an event loop. Use `AsyncFishAudio` and `await` the call: ```python theme={null} import asyncio from fishaudio import AsyncFishAudio async def main(): client = AsyncFishAudio() # reads FISH_API_KEY with open("speech.wav", "rb") as f: result = await client.asr.transcribe(audio=f.read(), language="en") print(result.text) asyncio.run(main()) ``` To run several files in parallel, gather the coroutines: ```python theme={null} import asyncio from fishaudio import AsyncFishAudio async def transcribe_all(paths): client = AsyncFishAudio() clips = [open(p, "rb").read() for p in paths] return await asyncio.gather(*[ client.asr.transcribe(audio=clip, language="en") for clip in clips ]) for result in asyncio.run(transcribe_all(["speech.wav"])): print(result.text) ``` ### Direct API (MessagePack) `POST /v1/asr` also accepts a [MessagePack](https://msgpack.org) body instead of multipart form data — the same path the API reference links to for low-overhead, server-side calls. Pack the audio bytes and options into one payload and set `Content-Type: application/msgpack`: ```python theme={null} import os import httpx import ormsgpack with open("speech.wav", "rb") as f: audio = f.read() payload = {"audio": audio, "language": "en", "ignore_timestamps": False} resp = httpx.post( "https://api.fish.audio/v1/asr", content=ormsgpack.packb(payload), headers={ "Authorization": f"Bearer {os.environ['FISH_API_KEY']}", "Content-Type": "application/msgpack", }, ) result = resp.json() print(result["text"]) ``` The response shape is identical to the multipart path: `text`, `duration` (seconds), and `segments`. ## Going further The reverse direction — text to lifelike audio. Every field and the raw response schema. `asr.transcribe` options and the `ASRResponse` type. # Text to Speech Source: https://docs.fish.audio/features/text-to-speech Turn text into lifelike speech — use it however you build Generate natural speech from text with the `s2.1-pro`, `s2-pro`, and `s1` models. Pick a voice, choose a format, and go — from the API directly, the Python library, or JavaScript. No code — type, pick a voice, generate. Every parameter for `POST /v1/tts`. Ready-made recipes: streaming, telephony, and more. ## When to use it Audiobooks, explainers, ads, and video narration. Speak an assistant's replies — pair with [streaming](/features/realtime-streaming) for low latency. Read content aloud, phone menus, notifications. Speak in a [cloned voice](/features/voice-cloning) you own. ## Quick start Send text, get back audio. Choose your implementation: ```python Python theme={null} from fishaudio import FishAudio from fishaudio.utils import save client = FishAudio() # reads FISH_API_KEY audio = client.tts.convert(text="Hello from Fish Audio!") save(audio, "out.mp3") ``` ```bash API (curl) theme={null} curl --request POST https://api.fish.audio/v1/tts \ --header "Authorization: Bearer $FISH_API_KEY" \ --header "Content-Type: application/json" \ --header "model: s2-pro" \ --data '{ "text": "Hello from Fish Audio!", "format": "mp3" }' \ --output out.mp3 ``` ```javascript JavaScript theme={null} import { FishAudioClient } from "fish-audio"; import { writeFile } from "fs/promises"; const client = new FishAudioClient({ apiKey: process.env.FISH_API_KEY }); // `s2-pro` is passed explicitly (the SDK default is `s1`). const stream = await client.textToSpeech.convert( { text: "Hello from Fish Audio!" }, "s2-pro", ); const chunks = []; for await (const chunk of stream) chunks.push(Buffer.from(chunk)); await writeFile("hello.mp3", Buffer.concat(chunks)); ``` ## Use a specific voice Pass a **voice model id** (`reference_id`). Find ids in the [Voice Library](/overview/platform) or create your own via [Voice Cloning](/features/voice-cloning). ```python Python theme={null} audio = client.tts.convert( text="This uses a specific voice.", reference_id="802e3bc2b27e49c2995d23ef70e6ac89", ) ``` ```bash API (curl) theme={null} curl --request POST https://api.fish.audio/v1/tts \ --header "Authorization: Bearer $FISH_API_KEY" \ --header "Content-Type: application/json" \ --header "model: s2-pro" \ --data '{ "text": "This uses a specific voice.", "reference_id": "802e3bc2b27e49c2995d23ef70e6ac89", "format": "mp3" }' \ --output out.mp3 ``` ## Implementation details ### Models * **`s2.1-pro`** — recommended for production, with improved quality, latency, and throughput over S2-Pro. * **`s2.1-pro-free`** — the same model at \$0 for testing, prototyping, development, and smaller businesses, without TTFA or DPA guarantees. * **`s2-pro`** (default) — previous-generation S2 model with multi-speaker and natural-language expression control. * **`s1`** — previous generation, `(parenthesis)` emotion tags. In the API, select with the `model` request header. In Python, pass `model="s2-pro"`. See [Choosing a Model](/developer-guide/models-pricing/choosing-a-model). ### Output formats `mp3` (default), `wav`, `pcm`, `opus`. Set `format` (and optionally `mp3_bitrate`, `sample_rate`). ```python Python theme={null} from fishaudio.types import TTSConfig audio = client.tts.convert( text="High quality", config=TTSConfig(format="wav", sample_rate=44100), ) ``` ```bash API (curl) theme={null} curl --request POST https://api.fish.audio/v1/tts \ --header "Authorization: Bearer $FISH_API_KEY" \ --header "Content-Type: application/json" \ --header "model: s2-pro" \ --data '{ "text": "High quality", "format": "wav", "sample_rate": 44100 }' \ --output out.wav ``` ### Speed & prosody Adjust speech speed (0.5–2.0) and volume. ```python Python theme={null} audio = client.tts.convert(text="Speaking faster.", speed=1.5) ``` ```bash API (curl) theme={null} curl --request POST https://api.fish.audio/v1/tts \ --header "Authorization: Bearer $FISH_API_KEY" \ --header "Content-Type: application/json" \ --header "model: s2-pro" \ --data '{ "text": "Speaking faster.", "prosody": { "speed": 1.5 } }' \ --output out.mp3 ``` ### Generation methods (Python) The Python SDK exposes three ways to generate, depending on whether you have the full text upfront and how you want to consume the audio: | Method | Returns | Use it for | | ------------------------ | ----------------------------------------------- | ----------------------------------------------------------------------------- | | `tts.convert()` | complete audio `bytes` | most cases — you have the text, you want the file | | `tts.stream()` | `AudioStream` (iterate chunks, or `.collect()`) | memory-efficient transfer of large audio; write chunks to disk as they arrive | | `tts.stream_websocket()` | iterator of audio `bytes` | text arriving in real time (LLM tokens, live captions) | ```python theme={null} # Memory-efficient: write each chunk as it arrives instead of buffering audio_stream = client.tts.stream(text="A very long passage...") with open("out.mp3", "wb") as f: for chunk in audio_stream: f.write(chunk) ``` For real-time text streaming with `stream_websocket()`, see [Realtime Streaming](/features/realtime-streaming). ### Instant voice cloning (reference audio) Instead of a saved `reference_id`, pass raw audio plus its transcript to clone a voice on the fly — no training step. Best with a clean 10–30s sample. ```python theme={null} from fishaudio.types import ReferenceAudio with open("sample.wav", "rb") as f: audio = client.tts.convert( text="Spoken in the reference voice.", references=[ReferenceAudio(audio=f.read(), text="Transcript of the sample.")], ) ``` To reuse a voice across many requests, [clone it once](/features/voice-cloning) and pass the resulting `reference_id` instead. ### Format & bitrate Pick a format for your delivery channel, and tune bitrate to trade size against quality: | Format | Notes | | --------------- | ---------------------------------------------------------------------------- | | `mp3` (default) | good size/quality balance; set `mp3_bitrate` to `64`, `128`, or `192` | | `wav` | uncompressed, highest quality; set `sample_rate` (e.g. `44100`) | | `pcm` | raw samples, no container — for low-latency playback and telephony pipelines | | `opus` | efficient for streaming; bitrate is automatic (`opus_bitrate=-1000`) | ```python theme={null} from fishaudio.types import TTSConfig audio = client.tts.convert( text="Smaller file, lower bitrate.", config=TTSConfig(format="mp3", mp3_bitrate=64), ) ``` ### Latency & chunk length `latency` trades stability for speed; `chunk_length` controls how much text the engine batches before it starts generating. * `latency="balanced"` (default) — lower time-to-first-audio (\~300ms). Good for interactive use. * `latency="normal"` — most stable output, at slightly higher latency. * `chunk_length` (`100`–`300`, default `200`) — smaller chunks start audio sooner; larger chunks are more efficient for long text. ```python Python theme={null} from fishaudio.types import TTSConfig audio = client.tts.convert( text="Quick, responsive output.", config=TTSConfig(latency="balanced", chunk_length=150), ) ``` ```bash API (curl) theme={null} curl --request POST https://api.fish.audio/v1/tts \ --header "Authorization: Bearer $FISH_API_KEY" \ --header "Content-Type: application/json" \ --header "model: s2-pro" \ --data '{ "text": "Quick, responsive output.", "latency": "balanced", "chunk_length": 150 }' \ --output out.mp3 ``` ### Direct API (MessagePack) `POST /v1/tts` also accepts a MessagePack body (`Content-Type: application/msgpack`) — the path the [API reference](/api-reference/endpoint/openapi-v1/text-to-speech) is built around. Use it to send binary reference audio in the request without base64 overhead, or when you don't want the SDK. ```python theme={null} import os import httpx import ormsgpack payload = {"text": "Hello from the direct API.", "reference_id": "YOUR_VOICE_ID", "format": "mp3"} resp = httpx.post( "https://api.fish.audio/v1/tts", content=ormsgpack.packb(payload), headers={ "Authorization": f"Bearer {os.environ['FISH_API_KEY']}", "Content-Type": "application/msgpack", "model": "s2-pro", }, ) with open("out.mp3", "wb") as f: f.write(resp.content) ``` The `model` header is required on every request. JSON and MessagePack accept the same fields. ### Advanced generation tuning For finer control, `TTSConfig` exposes the model's sampling parameters. The defaults are well-tuned — reach for these only when you need to dial in determinism or curb artifacts. ```python theme={null} from fishaudio.types import TTSConfig, Prosody config = TTSConfig( prosody=Prosody(speed=1.1, volume=0), temperature=0.7, # lower = more deterministic top_p=0.7, repetition_penalty=1.2, # >1.0 curbs repeated sounds max_new_tokens=1024, # cap audio length per chunk normalize=True, # expand numbers/dates for natural reading ) audio = client.tts.convert(text="Carefully tuned output.", config=config) ``` A `TTSConfig` is reusable — define it once and pass it to many `convert()` calls. See the [full field list](/api-reference/sdk/python/types#ttsconfig-objects) for every parameter and default. ## Going further Lowest latency for conversational and live apps. Direct delivery with tags and prosody. Every field, type, and default. `tts.convert` / `stream` / `stream_websocket`. # Voice Cloning Source: https://docs.fish.audio/features/voice-cloning Create a custom voice from audio samples, then speak with it Build a reusable voice model from your own audio, then use it anywhere you generate speech. You get back a voice **id** — pass it as `reference_id` to [Text to Speech](/features/text-to-speech) and every generation speaks in that voice. Works from the API directly, the Python library, or JavaScript. No code — clone a voice in the browser. Every field for `POST /model`. Instant clones, training, and reuse. ## When to use it One consistent voice across product, ads, and IVR. Clone your own voice for narration or assistants. Distinct voices for games, stories, and dialogue. Keep a speaker's identity across languages. ## Quick start Send one or more audio samples, get back a voice model. Choose your implementation: ```python Python theme={null} from fishaudio import FishAudio client = FishAudio() # reads FISH_API_KEY with open("sample.wav", "rb") as f: voice = client.voices.create( title="My Voice", voices=[f.read()], description="Cloned from a studio sample", visibility="private", ) print(voice.id, voice.state) ``` ```bash API (curl) theme={null} curl --request POST https://api.fish.audio/model \ --header "Authorization: Bearer $FISH_API_KEY" \ --form type=tts \ --form title="My Voice" \ --form "description=Cloned from a studio sample" \ --form visibility=private \ --form train_mode=fast \ --form voices=@sample.wav # Returns the new model, including its "_id" and "state". ``` ```javascript JavaScript theme={null} import { FishAudioClient } from "fish-audio"; import { readFile } from "fs/promises"; const client = new FishAudioClient({ apiKey: process.env.FISH_API_KEY }); const sample = await readFile("reference.wav"); const voice = await client.voices.ivc.create({ title: "My Voice", voices: [new File([sample], "reference.wav")], description: "Cloned from a studio sample", visibility: "private", }); console.log(voice._id, voice.state); ``` ## Use your cloned voice Pass the voice **id** as `reference_id` to Text to Speech — exactly like any other voice. ```python Python theme={null} audio = client.tts.convert( text="Now I speak in my cloned voice.", reference_id=voice.id, ) ``` ```bash API (curl) theme={null} curl --request POST https://api.fish.audio/v1/tts \ --header "Authorization: Bearer $FISH_API_KEY" \ --header "Content-Type: application/json" \ --header "model: s2-pro" \ --data '{ "text": "Now I speak in my cloned voice.", "reference_id": "YOUR_VOICE_ID" }' \ --output out.mp3 ``` ## Implementation details ### Sample quality Clean, mono, single-speaker audio gives the best result. A short clip works for a quick clone; a minute or two of clear speech improves fidelity. Avoid background music, reverb, and overlapping voices. ### Multiple samples Pass several clips to capture more range. You can also supply the matching transcripts as `texts` to sharpen pronunciation. ```python Python theme={null} voice = client.voices.create( title="My Voice", voices=[open("a.wav", "rb").read(), open("b.wav", "rb").read()], texts=["Transcript of clip A.", "Transcript of clip B."], ) ``` ```bash API (curl) theme={null} curl --request POST https://api.fish.audio/model \ --header "Authorization: Bearer $FISH_API_KEY" \ --form type=tts \ --form title="My Voice" \ --form voices=@a.wav \ --form voices=@b.wav ``` ### Visibility Models are `private` by default. Set `unlist` for a shareable link, or `public` to publish to the [Voice Library](/overview/platform). You can change this later — see [Manage Voices](/features/manage-voices). ## Instant vs. persistent clones There are two ways to clone: * **Persistent model** (above) — train once with `voices.create()`, get back a reusable `id`. Best when you'll use the same voice repeatedly. * **Instant clone** — pass reference audio inline on each generation with no model to manage. Best for one-off or per-request voices. For an instant clone, send the reference audio (and its transcript) directly to Text to Speech via `references` instead of `reference_id`: ```python Python theme={null} from fishaudio import FishAudio from fishaudio.types import ReferenceAudio client = FishAudio() with open("reference.wav", "rb") as f: audio = client.tts.convert( text="This will sound like the reference voice.", references=[ReferenceAudio( audio=f.read(), text="The exact words spoken in the reference clip.", )], ) ``` Pass several `ReferenceAudio` entries to capture more range, just as you would with multiple samples in a persistent model. The matching `text` for each clip sharpens pronunciation. ## Sample audio requirements Samples can be `.wav`, `.mp3`, `.m4a`, or `.opus`. Aim for at least 10 seconds per clip; a minute or two of clear, single-speaker speech improves fidelity. `enhance_audio_quality` (on by default) removes background noise and normalizes levels before training: ```python Python theme={null} voice = client.voices.create( title="My Voice", voices=[open("sample.wav", "rb").read()], enhance_audio_quality=True, ) ``` Leave it on for noisy or lower-quality recordings. If your audio is already clean and studio-grade, turning it off (`enhance_audio_quality=False`) avoids any extra processing. ## Model state A new model reports a `state` field that moves from `created` to `trained` (or `failed`). With `train_mode="fast"` (the default) the voice is usable almost immediately, so most clones return already `trained`. ```python Python theme={null} voice = client.voices.create(title="My Voice", voices=[sample]) print(voice.state) # "trained" ``` If a generation rejects the `reference_id`, re-fetch the model and confirm its state before using it in Text to Speech: ```python Python theme={null} voice = client.voices.get(voice.id) if voice.state == "trained": audio = client.tts.convert(text="Hello.", reference_id=voice.id) ``` ## Going further Use `reference_id` in any generation. List, update, and delete your voice models. Get the most natural results from your samples. Every field for `POST /model`. # Voice Design Source: https://docs.fish.audio/features/voice-design Generate candidate voices from a prompt Voice Design creates short voice candidates from a natural-language prompt. Use it when you want to explore a voice direction before building a longer text-to-speech workflow or creating a persistent voice model. Every parameter for `POST /v1/voice-design`. Voice Design is billed per successful generation request. Create a reusable voice model from reference audio. ## When to use it Generate several candidate voices from a short creative brief. Provide preview text to hear how a generated voice reads a specific line. Use generated candidates to choose a voice direction before longer TTS production. Get generated audio directly without creating batches, samples, or voice models. ## Quick start Send a JSON request with a prompt and receive generated candidates. The current candidate audio payload is WAV bytes encoded as base64. ```bash API (curl) theme={null} curl --request POST https://api.fish.audio/v1/voice-design \ --header "Authorization: Bearer $FISH_API_KEY" \ --header "Content-Type: application/json" \ --header "model: voice-design-1" \ --data '{ "instruction": "Warm, confident studio narrator with a natural tone", "reference_text": "Welcome to Fish Audio.", "language": "en", "n": 2 }' | jq -r '.candidates[0].audio_base64' | base64 --decode > voice.wav ``` ```python Python theme={null} import base64 import os import requests response = requests.post( "https://api.fish.audio/v1/voice-design", headers={ "Authorization": f"Bearer {os.environ['FISH_API_KEY']}", "Content-Type": "application/json", "model": "voice-design-1", }, json={ "instruction": "Warm, confident studio narrator with a natural tone", "reference_text": "Welcome to Fish Audio.", "language": "en", "n": 2, }, timeout=120, ) response.raise_for_status() candidate = response.json()["candidates"][0] with open("voice.wav", "wb") as f: f.write(base64.b64decode(candidate["audio_base64"])) print(candidate["sample_rate"], candidate["duration_ms"]) ``` ```javascript JavaScript theme={null} import { writeFile } from "node:fs/promises"; const response = await fetch("https://api.fish.audio/v1/voice-design", { method: "POST", headers: { Authorization: `Bearer ${process.env.FISH_API_KEY}`, "Content-Type": "application/json", model: "voice-design-1", }, body: JSON.stringify({ instruction: "Warm, confident studio narrator with a natural tone", reference_text: "Welcome to Fish Audio.", language: "en", n: 2, }), }); if (!response.ok) throw new Error(`${response.status} ${await response.text()}`); const { candidates } = await response.json(); await writeFile("voice.wav", Buffer.from(candidates[0].audio_base64, "base64")); console.log(candidates[0].sample_rate, candidates[0].duration_ms); ``` ## Prompt and preview text `instruction` is the main voice design prompt. Describe the voice, age, delivery, tone, accent, pacing, and context in natural language. ```json theme={null} { "instruction": "Energetic young presenter, bright tone, crisp diction, friendly but not cartoonish", "reference_text": "Here is your weekly product update.", "language": "en", "n": 3 } ``` `reference_text` is optional. When you provide it, candidates read that text so you can compare voices on the same line. Keep it short; the API accepts up to 300 characters. ## Parameters | Field | Default | Notes | | ------------------------- | -------- | ------------------------------------------------------------------ | | `instruction` | Required | Voice design prompt. 1 to 2000 characters. | | `reference_text` | `null` | Optional preview text. Up to 300 characters. | | `language` | `null` | Optional language hint such as `en`, `zh`, or `ja`. | | `n` | `2` | Number of candidates to generate. Range: 1 to 4. | | `speed` | `1.0` | Speaking speed multiplier. Must be greater than 0 and at most 3. | | `num_step` | `32` | Diffusion steps. Range: 1 to 128. | | `guidance_scale` | `2.0` | Higher values follow the prompt more strongly. Must be at least 0. | | `instruct_guidance_scale` | `0.0` | Prompt conditioning guidance. Must be at least 0. | | `seed` | `null` | Optional deterministic seed for candidate generation. | Voice Design accepts JSON only. Do not send MessagePack, multipart form data, inline reference audio, or service-internal fields such as `features`, `features_json_file`, or `include_audio_base64`. ## Response The response contains one or more generated candidates: ```json theme={null} { "candidates": [ { "id": "candidate-id", "index": 0, "audio_base64": "UklGRg...", "sample_rate": 44100, "duration_ms": 3100, "text": "Welcome to Fish Audio.", "language": "en" } ] } ``` Use `index` to preserve the order returned by the model. `id` is a stable candidate identifier for this response. Optional fields such as `text`, `instruct`, and `language` appear only when available. ## Billing and errors Voice Design is billed once per successful generation request, not once per candidate. Authentication errors, validation errors, insufficient API credit, concurrency limits, upstream service errors, and empty candidate responses are not billed. For the full error format and retry guidance, see [Errors](/api-reference/errors). # Overview Source: https://docs.fish.audio/overview/capabilities Everything Fish Audio can do — and how to build with it Fish Audio is a voice AI platform. Every core feature is available three ways: in the [web app](/overview/platform) (no code), through the [REST API](/api-reference/introduction), and via the official [SDK](/developer-guide/sdk-guide/quickstart). ## Core features Convert text into lifelike speech with the `s2.1-pro`, `s2-pro`, and `s1` models. Transcribe audio to text with per-segment timestamps. Clone a voice instantly from a clip, or train a persistent model. Stream audio as it generates — for voice agents and live apps. List, inspect, update, and delete your voice models. ## Also in the web app These run in the browser, no code required — see the [Platform guide](/overview/platform). Transform existing audio into a different voice. Produce multi-speaker, long-form audio — audiobooks and narration. Generate music and cinematic sound effects from a prompt. Split audio into stems, and related processing utilities. ## Models These text-to-speech models power most capabilities: * **`s2.1-pro`** — the recommended production model, with improved quality, latency, and throughput over S2-Pro. * **`s2.1-pro-free`** — the same model at \$0 for testing, prototyping, development, and smaller businesses, without TTFA or DPA guarantees. * **`s2-pro`** — the previous-generation S2 model, with multi-speaker and natural-language expression control. * **`s1`** — the previous generation, with `(parenthesis)` emotion tags. See [Models Overview](/developer-guide/models-pricing/models-overview) and [Choosing a Model](/developer-guide/models-pricing/choosing-a-model) for the full lineup, languages, and limits. ## Pick your path No code — generate audio, clone voices, and produce projects in your browser. The Python library for your application. Raw REST and WebSocket endpoints for any language. Install the Fish Audio skill so your agent writes correct code. # Platform (Web App) Source: https://docs.fish.audio/overview/platform Use Fish Audio in your browser — no code required The [Fish Audio web app](https://fish.audio/app) gives you every capability without writing code: generate speech, clone voices, produce long-form projects, and manage your account. Sign in at [fish.audio/app](https://fish.audio/app). This page is an orientation map of the web app. Detailed, step-by-step walkthroughs for each tool are coming. To build with code instead, see the [SDK Quickstart](/developer-guide/sdk-guide/quickstart) or the [API Reference](/api-reference/introduction). ## Create audio Type or paste text, pick a voice and model, and generate speech. Upload audio and transform it into another voice. Upload audio to transcribe it, with timestamps. Generate cinematic sound effects from a text prompt. Generate music from a description. Split audio into stems (e.g. vocals and background). ## Voices Create a custom voice from your own audio samples. Browse and use thousands of community and official voices. Manage the voice models you've created or saved. ## Produce projects Assemble multi-speaker, long-form audio — audiobooks, dialogue, and narration — in a project editor. ## Library & history Find, replay, and download everything you've generated. Your saved audio, models, and collections. ## Account & billing Create and manage keys for the API and SDKs. Subscription, credits, and invoices. See [Pricing & Rate Limits](/developer-guide/models-pricing/pricing-and-rate-limits). Track your consumption. Shared workspaces, members, and billing for organizations.