Skip to main content
Fish Audio provides Docker images for both WebUI and API server deployments. You can use pre-built images from Docker Hub or build custom images locally.

Prerequisites

Before deploying with Docker, ensure you have:
  • Docker and Docker Compose installed
  • NVIDIA Docker runtime (for GPU support)
  • At least 12GB GPU memory for CUDA inference
  • Downloaded model weights (see Running Inference)

Pre-built Images

Fish Audio provides ready-to-use Docker images on Docker Hub:
ImageDescriptionBest For
fishaudio/fish-speech:latest-webui-cudaWebUI with CUDA supportInteractive development with GPU
fishaudio/fish-speech:latest-webui-cpuWebUI CPU-onlyTesting without GPU
fishaudio/fish-speech:latest-server-cudaAPI server with CUDAProduction deployments with GPU
fishaudio/fish-speech:latest-server-cpuAPI server CPU-onlyLow-traffic CPU deployments
For production use, we recommend using specific version tags instead of latest to ensure consistency across deployments.

Quick Start with Docker Run

The fastest way to get started is using docker run:

WebUI Deployment

# Create directories for model weights and reference audio
mkdir -p checkpoints references

# Start WebUI with CUDA support (recommended)
docker run -d \
    --name fish-speech-webui \
    --gpus all \
    -p 7860:7860 \
    -v ./checkpoints:/app/checkpoints \
    -v ./references:/app/references \
    -e COMPILE=1 \
    fishaudio/fish-speech:latest-webui-cuda

# For CPU-only deployment
docker run -d \
    --name fish-speech-webui-cpu \
    -p 7860:7860 \
    -v ./checkpoints:/app/checkpoints \
    -v ./references:/app/references \
    fishaudio/fish-speech:latest-webui-cpu
Access the WebUI at http://localhost:7860

API Server Deployment

# Start API server with CUDA support
docker run -d \
    --name fish-speech-server \
    --gpus all \
    -p 8080:8080 \
    -v ./checkpoints:/app/checkpoints \
    -v ./references:/app/references \
    -e COMPILE=1 \
    fishaudio/fish-speech:latest-server-cuda

# For CPU-only deployment
docker run -d \
    --name fish-speech-server-cpu \
    -p 8080:8080 \
    -v ./checkpoints:/app/checkpoints \
    -v ./references:/app/references \
    fishaudio/fish-speech:latest-server-cpu
Access the API documentation at http://localhost:8080
Enable the COMPILE=1 environment variable for ~10x faster inference on CUDA deployments. This uses torch.compile to optimize the model.

Docker Compose Deployment

For development or customization, Docker Compose provides easier configuration management:

Setup

# Clone the repository
git clone https://github.com/fishaudio/fish-speech.git
cd fish-speech

Start Services

# Start WebUI with CUDA
docker compose --profile webui up

# Start WebUI with compile optimization
COMPILE=1 docker compose --profile webui up

# Start API server
docker compose --profile server up

# Start API server with compile optimization
COMPILE=1 docker compose --profile server up

# For CPU-only deployment
BACKEND=cpu docker compose --profile webui up
Run containers in detached mode by adding the -d flag: docker compose --profile webui up -d

Environment Variables

Customize deployment using environment variables or a .env file:
# .env file example
BACKEND=cuda              # or cpu
COMPILE=1                 # Enable compile optimization
GRADIO_PORT=7860         # WebUI port
API_PORT=8080            # API server port
UV_VERSION=0.8.15        # UV package manager version

Manual Docker Build

For advanced users who need custom configurations:

Build WebUI Image

# Build with CUDA support
docker build \
    --platform linux/amd64 \
    -f docker/Dockerfile \
    --build-arg BACKEND=cuda \
    --build-arg CUDA_VER=12.6.0 \
    --build-arg UV_EXTRA=cu126 \
    --target webui \
    -t fish-speech-webui:cuda .

# Build CPU-only (supports multi-platform)
docker build \
    --platform linux/amd64,linux/arm64 \
    -f docker/Dockerfile \
    --build-arg BACKEND=cpu \
    --target webui \
    -t fish-speech-webui:cpu .

Build API Server Image

# Build with CUDA support
docker build \
    --platform linux/amd64 \
    -f docker/Dockerfile \
    --build-arg BACKEND=cuda \
    --build-arg CUDA_VER=12.6.0 \
    --build-arg UV_EXTRA=cu126 \
    --target server \
    -t fish-speech-server:cuda .

Build Development Image

# Build development image with all tools
docker build \
    --platform linux/amd64 \
    -f docker/Dockerfile \
    --build-arg BACKEND=cuda \
    --target dev \
    -t fish-speech-dev:cuda .

Build Arguments

ArgumentOptionsDefaultDescription
BACKENDcuda, cpucudaCompute backend
CUDA_VER12.6.0, etc.12.6.0CUDA version
UV_EXTRAcu126, cu128, cu129cu126UV extra for CUDA
UBUNTU_VER24.04, etc.24.04Ubuntu base version
PY_VER3.12, etc.3.12Python version

Volume Mounts

Both Docker run and Compose methods require these volume mounts:
Host PathContainer PathPurpose
./checkpoints/app/checkpointsModel weights directory
./references/app/referencesReference audio files for voice cloning
Ensure model weights are downloaded and placed in the ./checkpoints directory before starting containers. See Running Inference for download instructions.

Environment Variables Reference

WebUI Configuration

VariableDefaultDescription
GRADIO_SERVER_NAME0.0.0.0WebUI server host
GRADIO_SERVER_PORT7860WebUI server port
GRADIO_SHAREfalseEnable Gradio public sharing

API Server Configuration

VariableDefaultDescription
API_SERVER_NAME0.0.0.0API server host
API_SERVER_PORT8080API server port

Model Configuration

VariableDefaultDescription
LLAMA_CHECKPOINT_PATHcheckpoints/openaudio-s1-miniPath to model weights
DECODER_CHECKPOINT_PATHcheckpoints/openaudio-s1-mini/codec.pthPath to decoder weights
DECODER_CONFIG_NAMEmodded_dac_vqDecoder configuration name

Performance Optimization

VariableDefaultDescription
COMPILE0Enable torch.compile for ~10x speedup (CUDA only)

Container Management

View Logs

# Docker run
docker logs fish-speech-webui

# Docker Compose
docker compose logs webui

Stop Containers

# Docker run
docker stop fish-speech-webui

# Docker Compose
docker compose down

Update Images

# Pull latest images
docker pull fishaudio/fish-speech:latest-webui-cuda

# Restart containers with new image
docker compose --profile webui up -d

GPU Support

Prerequisites

Install NVIDIA Container Toolkit:
# Ubuntu/Debian
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | \
    sudo tee /etc/apt/sources.list.d/nvidia-docker.list

sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit
sudo systemctl restart docker

Verify GPU Access

docker run --rm --gpus all nvidia/cuda:12.6.0-base-ubuntu24.04 nvidia-smi
GPU support requires NVIDIA Docker runtime. For CPU-only deployment, remove the --gpus all flag and use CPU images.

Troubleshooting

Container Won’t Start

Check logs for errors:
docker logs fish-speech-webui
Common issues:
  • Missing model weights in ./checkpoints
  • Port already in use (change port mapping)
  • Insufficient GPU memory

GPU Not Detected

Verify NVIDIA Docker runtime is installed:
docker run --rm --gpus all nvidia/cuda:12.6.0-base-ubuntu24.04 nvidia-smi

Performance Issues

  1. Enable compile optimization: COMPILE=1
  2. Ensure GPU is being used (check with nvidia-smi)
  3. Verify sufficient GPU memory is available

Next Steps

I