Skip to main content
Fish Audio provides Docker images for both WebUI and API server deployments. You can use pre-built images from Docker Hub or build custom images locally.

​
Prerequisites

Before deploying with Docker, ensure you have:
  • Docker and Docker Compose installed
  • NVIDIA Docker runtime (for GPU support)
  • At least 12GB GPU memory for CUDA inference
  • Downloaded model weights (see Running Inference)

​
Pre-built Images

Fish Audio provides ready-to-use Docker images on Docker Hub:
ImageDescriptionBest For
fishaudio/fish-speech:latest-webui-cudaWebUI with CUDA supportInteractive development with GPU
fishaudio/fish-speech:latest-webui-cpuWebUI CPU-onlyTesting without GPU
fishaudio/fish-speech:latest-server-cudaAPI server with CUDAProduction deployments with GPU
fishaudio/fish-speech:latest-server-cpuAPI server CPU-onlyLow-traffic CPU deployments
For production use, we recommend using specific version tags instead of latest to ensure consistency across deployments.

​
Quick Start with Docker Run

The fastest way to get started is using docker run:

​
WebUI Deployment

# Create directories for model weights and reference audio
mkdir -p checkpoints references

# Start WebUI with CUDA support (recommended)
docker run -d \
    --name fish-speech-webui \
    --gpus all \
    -p 7860:7860 \
    -v ./checkpoints:/app/checkpoints \
    -v ./references:/app/references \
    -e COMPILE=1 \
    fishaudio/fish-speech:latest-webui-cuda

# For CPU-only deployment
docker run -d \
    --name fish-speech-webui-cpu \
    -p 7860:7860 \
    -v ./checkpoints:/app/checkpoints \
    -v ./references:/app/references \
    fishaudio/fish-speech:latest-webui-cpu
Access the WebUI at http://localhost:7860

​
API Server Deployment

# Start API server with CUDA support
docker run -d \
    --name fish-speech-server \
    --gpus all \
    -p 8080:8080 \
    -v ./checkpoints:/app/checkpoints \
    -v ./references:/app/references \
    -e COMPILE=1 \
    fishaudio/fish-speech:latest-server-cuda

# For CPU-only deployment
docker run -d \
    --name fish-speech-server-cpu \
    -p 8080:8080 \
    -v ./checkpoints:/app/checkpoints \
    -v ./references:/app/references \
    fishaudio/fish-speech:latest-server-cpu
Access the API documentation at http://localhost:8080
Enable the COMPILE=1 environment variable for ~10x faster inference on CUDA deployments. This uses torch.compile to optimize the model.

​
Docker Compose Deployment

For development or customization, Docker Compose provides easier configuration management:

​
Setup

# Clone the repository
git clone https://github.com/fishaudio/fish-speech.git
cd fish-speech

​
Start Services

# Start WebUI with CUDA
docker compose --profile webui up

# Start WebUI with compile optimization
COMPILE=1 docker compose --profile webui up

# Start API server
docker compose --profile server up

# Start API server with compile optimization
COMPILE=1 docker compose --profile server up

# For CPU-only deployment
BACKEND=cpu docker compose --profile webui up
Run containers in detached mode by adding the -d flag: docker compose --profile webui up -d

​
Environment Variables

Customize deployment using environment variables or a .env file: 
# .env file example
BACKEND=cuda              # or cpu
COMPILE=1                 # Enable compile optimization
GRADIO_PORT=7860         # WebUI port
API_PORT=8080            # API server port
UV_VERSION=0.8.15        # UV package manager version

​
Manual Docker Build

For advanced users who need custom configurations:

​
Build WebUI Image

# Build with CUDA support
docker build \
    --platform linux/amd64 \
    -f docker/Dockerfile \
    --build-arg BACKEND=cuda \
    --build-arg CUDA_VER=12.6.0 \
    --build-arg UV_EXTRA=cu126 \
    --target webui \
    -t fish-speech-webui:cuda .

# Build CPU-only (supports multi-platform)
docker build \
    --platform linux/amd64,linux/arm64 \
    -f docker/Dockerfile \
    --build-arg BACKEND=cpu \
    --target webui \
    -t fish-speech-webui:cpu .

​
Build API Server Image

# Build with CUDA support
docker build \
    --platform linux/amd64 \
    -f docker/Dockerfile \
    --build-arg BACKEND=cuda \
    --build-arg CUDA_VER=12.6.0 \
    --build-arg UV_EXTRA=cu126 \
    --target server \
    -t fish-speech-server:cuda .

​
Build Development Image

# Build development image with all tools
docker build \
    --platform linux/amd64 \
    -f docker/Dockerfile \
    --build-arg BACKEND=cuda \
    --target dev \
    -t fish-speech-dev:cuda .

​
Build Arguments

ArgumentOptionsDefaultDescription
BACKENDcuda, cpucudaCompute backend
CUDA_VER12.6.0, etc.12.6.0CUDA version
UV_EXTRAcu126, cu128, cu129cu126UV extra for CUDA
UBUNTU_VER24.04, etc.24.04Ubuntu base version
PY_VER3.12, etc.3.12Python version

​
Volume Mounts

Both Docker run and Compose methods require these volume mounts:
Host PathContainer PathPurpose
./checkpoints/app/checkpointsModel weights directory
./references/app/referencesReference audio files for voice cloning
Ensure model weights are downloaded and placed in the ./checkpoints directory before starting containers. See Running Inference for download instructions.

​
Environment Variables Reference

​
WebUI Configuration

VariableDefaultDescription
GRADIO_SERVER_NAME0.0.0.0WebUI server host
GRADIO_SERVER_PORT7860WebUI server port
GRADIO_SHAREfalseEnable Gradio public sharing

​
API Server Configuration

VariableDefaultDescription
API_SERVER_NAME0.0.0.0API server host
API_SERVER_PORT8080API server port

​
Model Configuration

VariableDefaultDescription
LLAMA_CHECKPOINT_PATHcheckpoints/openaudio-s1-miniPath to model weights
DECODER_CHECKPOINT_PATHcheckpoints/openaudio-s1-mini/codec.pthPath to decoder weights
DECODER_CONFIG_NAMEmodded_dac_vqDecoder configuration name

​
Performance Optimization

VariableDefaultDescription
COMPILE0Enable torch.compile for ~10x speedup (CUDA only)

​
Container Management

​
View Logs

# Docker run
docker logs fish-speech-webui

# Docker Compose
docker compose logs webui

​
Stop Containers

# Docker run
docker stop fish-speech-webui

# Docker Compose
docker compose down

​
Update Images

# Pull latest images
docker pull fishaudio/fish-speech:latest-webui-cuda

# Restart containers with new image
docker compose --profile webui up -d

​
GPU Support

​
Prerequisites

Install NVIDIA Container Toolkit: 
# Ubuntu/Debian
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | \
    sudo tee /etc/apt/sources.list.d/nvidia-docker.list

sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit
sudo systemctl restart docker

​
Verify GPU Access

docker run --rm --gpus all nvidia/cuda:12.6.0-base-ubuntu24.04 nvidia-smi
GPU support requires NVIDIA Docker runtime. For CPU-only deployment, remove the --gpus all flag and use CPU images.

​
Troubleshooting

​
Container Won’t Start

Check logs for errors: 
docker logs fish-speech-webui
Common issues:
  • Missing model weights in ./checkpoints
  • Port already in use (change port mapping)
  • Insufficient GPU memory

​
GPU Not Detected

Verify NVIDIA Docker runtime is installed: 
docker run --rm --gpus all nvidia/cuda:12.6.0-base-ubuntu24.04 nvidia-smi

​
Performance Issues

  1. Enable compile optimization: COMPILE=1
  2. Ensure GPU is being used (check with nvidia-smi)
  3. Verify sufficient GPU memory is available

​
Next Steps