Skip to main content
Every reply tries the agent’s own model first. That is the model in llm.model or llm.custom on the configuration. Fallback decides what happens when that attempt produces no reply:
  • The provider returns an error.
  • The model takes too long.
  • The response stream breaks off with nothing in it.
Instead of the call going silent, the platform retries the reply on another model. A switch only affects the reply that failed. The next reply tries the agent’s own model again.

​
Modes

llm.fallback.mode picks the behavior: With manual you choose which models back up the agent, in what order, and with what timing.

​
Configure manual fallback

Set llm.fallback on the agent’s draft configuration through PATCH /v1/agent/agents/{agent_id}/config. It works like any other llm field.
Changes take effect for new sessions after you publish.

​
models

1 to 3 entries, tried in the order you list them. Each entry names one model: Rules:
  • An entry carries model or custom, not both. Anything else is 422.
  • No duplicates. The main model and every entry must be different models or endpoints. A duplicate is 422.
  • A custom main model can fall back to platform models.
  • A platform main model can fall back to a custom endpoint.
  • api_key is write-only. Reads and version snapshots return null.
  • Omit api_key on a later patch to keep the stored key for the same base_url and model.

​
Timeouts

Each entry in models can set two timeouts. primary sets the same two for the agent’s own model. When total_timeout_ms runs out:
  • Nothing spoken yet: the reply moves to the next model.
  • Playback already started: the reply counts as failed. No switch.
These numbers are starting points, not thresholds. A shorter ttft_timeout_ms reaches a fallback sooner. It also switches away from a model that was about to answer.

​
Run a fallback in parallel

The first entry in models can set parallel.kick_in_ms. Range 300–10,000, default 1500.
  • The platform requests the agent’s own model and this fallback at the same time.
  • If the main model has no first token by kick_in_ms, the fallback’s reply is used.
  • The other request is cancelled.
  • Only the first entry can set parallel. On a later entry it is 422.
Parallel trades cost for latency. Both requests bill their own tokens, whichever reply is used.

​
What triggers a switch

A reply moves to the next model on:
  • A provider error.
  • No first token within ttft_timeout_ms. Outside manual, within the platform’s own limit.
  • total_timeout_ms reached before anything was spoken.
  • A response stream that breaks off with no output.

​
What doesn’t switch

  • The agent choosing silence. A model that finishes normally with an empty reply is the agent staying quiet on purpose. Nothing switches.
  • A reply already playing. Once the caller hears the answer, a problem finishing it counts as a failed reply.

​
Per-session override

A session can replace the agent’s fallback configuration for that conversation only. See overrides.llm.fallback.

​
See which model answered

The session detail endpoint reports every switch and which model wrote each reply. See Conversation History. The console’s Conversations view shows the same: each switch in the timeline, and which model answered each turn.

​
Billing

  • Each reply is billed by the model that answered it, at that model’s rate.
  • A reply from a custom LLM endpoint has no LLM charge. Main model or fallback.
  • The whole call is free of LLM charges only when the main model and every fallback are free.

​
Going further

Custom LLM

Point the agent, or a fallback entry, at your own endpoint.

Configuration

Where llm.fallback sits among the rest of the agent’s settings.

Authenticated sessions

Override the fallback configuration for a single session.

Conversation history

Inspect every switch after the call.