Skip to main content
Running the Helm platform once it is live. For the single-container platform, see All-in-One container. Commands and troubleshooting are in your deployment guide.

​
Ownership

Agree an escalation path before production traffic starts.

​
Monitoring signals

Prometheus is the metrics baseline. Watch request success, latency, and time-to-first-audio; pod readiness and restarts, Redis included; CPU and memory against requests and limits, and GPU use; shared storage; image pull failures; and the license’s time to expiry. The services export it as a metric and log warnings well before the license lapses. Centralize logs, keep Kubernetes events long enough to investigate a bad rollout, and redact secrets before sharing logs outside your environment.

​
Scaling

Capacity is set in the release configuration (replicas, resource requests, and per-worker concurrency) and applied with helm upgrade. Autoscaling on load is included and off by default; your deployment guide covers turning it on. Add a GPU replica only when a GPU is actually free: a pod that requests one on a full cluster stays Pending. Validate every change at your expected concurrency, and allow minutes for a new GPU worker to become ready.

​
Backup and retention

Back up:
  • Reference voice archives, if your traffic uses reference ids. In a self-hosted deployment these are durable data, not cache.
  • The offline usage ledger directories, before they are pruned by your own archival process.
  • The values file used for the production release.
  • Secret manager entries.
  • Dashboards and alert rules.
Compile and model caches on shared storage do not need backing up. Losing them costs a slow first start, nothing more.

​
Upgrades

Take the new version from Developer → Self Host and follow its deployment runbook, which covers rollback. Talk to your account team before skipping versions or upgrading production.

​
Escalating to Fish Audio

Include:
  • Kubernetes version, cloud provider, region, node types, and GPU type.
  • The model, its chart version, and the values files with secrets removed.
  • Pod status, relevant events, and logs with secrets redacted.
  • The exact command that failed, with its output.
  • Timestamp and time zone.