> ## Documentation Index > Fetch the complete documentation index at: https://docs.fish.audio/llms.txt > Use this file to discover all available pages before exploring further. # Create Test > Create a Single Turn, Tool or Simulation test, optionally attached to agents. ## OpenAPI ````yaml post /v1/agent/tests openapi: 3.1.0 info: title: FishAudio OpenAPI version: '1' servers: - description: Fish Audio API url: https://api.fish.audio security: [] tags: [] paths: /v1/agent/tests: post: tags: - Agent Tests summary: Create Test description: >- Create a test: `next_reply` judges the agent's next reply against an expectation, `tool` checks the tool the agent calls next, and `simulation` lets a simulated user hold a whole conversation that is scored against success conditions. Pass `agent_ids` to attach the test right away. The body of `GET /v1/agent/tests/{test_id}` posts back as is, so tests can be exported and imported. Tool references use your workspace's tool ids (list them with `GET /v1/agent/agents/{agent_id}/test-tools`), so an export imports cleanly within the same team and needs its tool ids remapped elsewhere. requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/PublicAgentTestCreatePayload' responses: '201': description: Document created, URL follows headers: {} content: application/json: schema: properties: test_id: title: Test Id type: string workspace_id: title: Workspace Id type: string name: title: Name type: string test_type: enum: - next_reply - tool - simulation title: Test Type type: string agent_ids: description: Agents this test is attached to. items: type: string title: Agent Ids type: array created_at: format: date-time title: Created At type: string updated_at: format: date-time title: Updated At type: string conversation: items: $ref: '#/components/schemas/PublicAgentTestMessage' title: Conversation type: array expectation: default: '' title: Expectation type: string success_examples: items: type: string title: Success Examples type: array failure_examples: items: type: string title: Failure Examples type: array referenced_tool: anyOf: - $ref: '#/components/schemas/AgentTestReferencedTool' - type: 'null' default: null tool_parameters: items: $ref: '#/components/schemas/AgentTestToolParameter' title: Tool Parameters type: array verify_absence: default: false title: Verify Absence type: boolean simulation: anyOf: - $ref: '#/components/schemas/AgentTestSimulationConfig' - type: 'null' default: null dynamic_variables: additionalProperties: anyOf: - type: string - type: integer - type: number - type: boolean title: Dynamic Variables type: object required: - test_id - workspace_id - name - test_type - created_at - updated_at type: object '400': description: Bad request syntax or unsupported method headers: {} content: application/json: schema: properties: status: title: Status type: integer message: title: Message type: string reason: anyOf: - type: string - type: 'null' default: null title: Reason required: - status - message type: object '401': description: No permission -- see authorization schemes headers: {} content: application/json: schema: properties: status: title: Status type: integer message: title: Message type: string reason: anyOf: - type: string - type: 'null' default: null title: Reason required: - status - message type: object '422': description: '' headers: {} content: application/json: schema: properties: status: title: Status type: integer message: title: Message type: string reason: anyOf: - type: string - type: 'null' default: null title: Reason required: - status - message type: object '503': description: The server cannot process the request due to a high load headers: {} content: application/json: schema: properties: status: title: Status type: integer message: title: Message type: string reason: anyOf: - type: string - type: 'null' default: null title: Reason required: - status - message type: object security: - BearerAuth: [] x-codeSamples: - lang: bash label: Create Simulation Test source: |- curl --request POST \ --url https://api.fish.audio/v1/agent/tests \ --header 'Authorization: Bearer ' \ --header 'Content-Type: application/json' \ --data '{ "name": "Reschedules an appointment", "test_type": "simulation", "agent_ids": [""], "simulation": { "scenario": "You are Jane. Move your Tuesday cleaning to Thursday afternoon.", "max_turns": 10, "success_conditions": [ {"name": "rescheduled", "description": "The agent confirms the new Thursday slot."} ], "tool_mocks": { "strategy": "all", "tools": [ { "tool": {"id": "", "name": "Book appointment", "type": "webhook"}, "result": {"status": "confirmed"} } ] } } }' components: schemas: PublicAgentTestCreatePayload: additionalProperties: false properties: name: maxLength: 200 minLength: 1 title: Name type: string test_type: default: next_reply enum: - next_reply - tool - simulation title: Test Type type: string conversation: items: $ref: '#/components/schemas/AgentTestMessagePayload' title: Conversation type: array expectation: default: '' maxLength: 400 title: Expectation type: string success_examples: items: type: string title: Success Examples type: array failure_examples: items: type: string title: Failure Examples type: array referenced_tool: anyOf: - $ref: '#/components/schemas/AgentTestReferencedTool' - type: 'null' default: null tool_parameters: items: $ref: '#/components/schemas/AgentTestToolParameter' title: Tool Parameters type: array verify_absence: default: false title: Verify Absence type: boolean simulation: anyOf: - $ref: '#/components/schemas/AgentTestSimulationConfig' - type: 'null' default: null dynamic_variables: additionalProperties: anyOf: - type: string - type: integer - type: number - type: boolean title: Dynamic Variables type: object agent_ids: description: >- Agents to attach the test to. They must all be in one workspace, and the test is created there. Without agents the test goes into the API key owner's default workspace. items: type: string maxItems: 100 title: Agent Ids type: array required: - name title: PublicAgentTestCreatePayload type: object PublicAgentTestMessage: properties: role: enum: - agent - user title: Role type: string text: title: Text type: string required: - role - text title: PublicAgentTestMessage type: object AgentTestReferencedTool: additionalProperties: false description: |- Tool the runner should (or should not) observe on the next turn. A webhook or client tool is referenced by its library id, an integration tool by ":" (for example "google_calendar:create_event"). properties: id: maxLength: 64 minLength: 1 title: Id type: string name: maxLength: 200 minLength: 1 title: Name type: string type: default: webhook enum: - webhook - client - integration title: Type type: string description: default: '' maxLength: 2000 title: Description type: string required: - id - name title: AgentTestReferencedTool type: object AgentTestToolParameter: additionalProperties: false properties: name: maxLength: 100 minLength: 1 title: Name type: string value: anyOf: - type: string - type: integer - type: number - type: boolean title: Value type: anyOf: - enum: - string - number - boolean type: string - type: 'null' default: null title: Type required: - name - value title: AgentTestToolParameter type: object AgentTestSimulationConfig: additionalProperties: false properties: scenario: maxLength: 10000 minLength: 1 title: Scenario type: string max_turns: default: 10 maximum: 50 minimum: 1 title: Max Turns type: integer success_conditions: items: $ref: '#/components/schemas/AgentTestSuccessCondition' maxItems: 10 minItems: 1 title: Success Conditions type: array assertions: $ref: '#/components/schemas/AgentTestSimulationAssertions' tool_mocks: $ref: '#/components/schemas/AgentTestToolMocks' simulated_user_model: anyOf: - type: string - type: 'null' default: null title: Simulated User Model repeat_count: default: 1 maximum: 20 minimum: 1 title: Repeat Count type: integer channel: anyOf: - enum: - web_voice - phone_inbound - phone_outbound type: string - type: 'null' default: null title: Channel required: - scenario - success_conditions title: AgentTestSimulationConfig type: object AgentTestMessagePayload: properties: role: enum: - agent - user title: Role type: string text: maxLength: 2000 minLength: 1 title: Text type: string required: - role - text title: AgentTestMessagePayload type: object AgentTestSuccessCondition: additionalProperties: false description: >- One natural-language criterion the judge scores on the whole conversation. properties: name: maxLength: 80 minLength: 1 title: Name type: string description: maxLength: 500 minLength: 1 title: Description type: string required: - name - description title: AgentTestSuccessCondition type: object AgentTestSimulationAssertions: additionalProperties: false description: Deterministic checks run before the judge, no LLM involved. properties: tool_calls: items: $ref: '#/components/schemas/AgentTestToolCallAssertion' maxItems: 100 title: Tool Calls type: array forbidden_tools: items: $ref: '#/components/schemas/AgentTestReferencedTool' maxItems: 100 title: Forbidden Tools type: array ended_by: anyOf: - enum: - agent - user - transfer - any type: string - type: 'null' default: null title: Ended By title: AgentTestSimulationAssertions type: object AgentTestToolMocks: additionalProperties: false description: |- all: every tool answers from a mock, except real_tools, which run for real. selected: only the listed tools do, the rest follow fallback. none: every tool hits its real endpoint. properties: strategy: default: all enum: - all - selected - none title: Strategy type: string fallback: default: error enum: - error - real title: Fallback type: string tools: items: $ref: '#/components/schemas/AgentTestToolMock' maxItems: 50 title: Tools type: array real_tools: items: $ref: '#/components/schemas/AgentTestReferencedTool' maxItems: 50 title: Real Tools type: array title: AgentTestToolMocks type: object AgentTestToolCallAssertion: additionalProperties: false description: |- The agent must have called this library tool, optionally with matching arguments, between min_calls and max_calls times (calls that match). properties: tool: $ref: '#/components/schemas/AgentTestReferencedTool' params: additionalProperties: $ref: '#/components/schemas/AgentTestParamMatcher' title: Params type: object min_calls: default: 1 maximum: 100 minimum: 0 title: Min Calls type: integer max_calls: anyOf: - maximum: 100 minimum: 0 type: integer - type: 'null' default: null title: Max Calls required: - tool title: AgentTestToolCallAssertion type: object AgentTestToolMock: additionalProperties: false description: >- A canned result for one library tool, optionally gated on the call arguments. properties: tool: $ref: '#/components/schemas/AgentTestReferencedTool' result: $ref: '#/components/schemas/JsonValue' default: null status: anyOf: - maximum: 599 minimum: 100 type: integer - type: 'null' default: null title: Status when: anyOf: - additionalProperties: $ref: '#/components/schemas/AgentTestParamMatcher' type: object - type: 'null' default: null title: When error: default: false title: Error type: boolean required: - tool title: AgentTestToolMock type: object AgentTestParamMatcher: additionalProperties: false description: |- How one tool argument is compared: exact string equality, a regular expression searched in the value, or mere presence. Regexes are JavaScript without flags, mock conditions run in the runtime as written and assertions are translated to Python for scoring. properties: type: default: exact enum: - exact - regex - any title: Type type: string value: anyOf: - maxLength: 1000 type: string - type: 'null' default: null title: Value title: AgentTestParamMatcher type: object JsonValue: {} securitySchemes: BearerAuth: type: http scheme: bearer ```` This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.