Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 24 additions & 3 deletions agents/build/custom-llm.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -58,16 +58,16 @@
"parameters": {
"type": "object",
"properties": {
"order_number": { "type": "string", "description": "The order number, e.g. A12345." }

Check warning on line 61 in agents/build/custom-llm.mdx

View check run for this annotation

Mintlify / Mintlify Validation (hanabiaiinc) - vale-spellcheck

agents/build/custom-llm.mdx#L61

Did you really mean 'order_number'?
},
"required": ["order_number"]
}
}
}
],
"session_id": "sess_01j9x4k2m8v3q7n5p6r8t9w0y1",

Check warning on line 68 in agents/build/custom-llm.mdx

View check run for this annotation

Mintlify / Mintlify Validation (hanabiaiinc) - vale-spellcheck

agents/build/custom-llm.mdx#L68

Did you really mean 'session_id'?
"user_id": "user-42",

Check warning on line 69 in agents/build/custom-llm.mdx

View check run for this annotation

Mintlify / Mintlify Validation (hanabiaiinc) - vale-spellcheck

agents/build/custom-llm.mdx#L69

Did you really mean 'user_id'?
"fishaudio_extra_body": { "chat_id": "chat-9" }

Check warning on line 70 in agents/build/custom-llm.mdx

View check run for this annotation

Mintlify / Mintlify Validation (hanabiaiinc) - vale-spellcheck

agents/build/custom-llm.mdx#L70

Did you really mean 'fishaudio_extra_body'?
}
```

Expand Down Expand Up @@ -95,7 +95,24 @@

## Point the agent at your endpoint

Set the `llm.custom` section on the agent's [configuration](/agents/build/configuration#configure-through-the-api):
To set it up in the console:

1. Open the agent's **Configuration** and click the **Model** selector.
2. Choose **+ Custom LLM** at the bottom of the menu.
3. Enter the base URL, the model name and your API key.
4. Click **Run test**. It sends one streaming turn with the agent's prompt and tools, just like a live call.
5. Click **Save and use**. The agent now uses your endpoint.

**Run test** reports:

- Whether the endpoint was reached.
- Whether it streamed `delta.content`.
- Whether it accepted the tools.
- The time to first token.

The saved endpoint appears under **Custom** in the selector. You can edit or remove it there. If you pick a platform model again, the endpoint and the stored key are removed.

Through the API, set the `llm.custom` section on the agent's [configuration](/agents/build/configuration#configure-through-the-api):

```bash
curl --request PATCH "https://api.fish.audio/v1/agent/agents/$AGENT_ID/config" \
Expand Down Expand Up @@ -129,11 +146,15 @@
- After three consecutive failed generations the agent apologizes, hangs up, and the session ends with `end_reason: llm_endpoint_failure` (in the [sessions API](/agents/monitor/conversation-history#status-and-end-reason), the dashboard, and the `call.ended` webhook's `ended_reason`). Each failure also emits an `llm.endpoint_error` event on the session timeline.

<Tip>
Voice conversations are latency sensitive, so aim for a time-to-first-token under 800 ms. Turn latency is attributed per session in [conversation history](/agents/monitor/conversation-history), which lets you tell endpoint time from platform time.
Voice conversations are latency sensitive:

- Aim for a time-to-first-token under 800 ms. This is a recommendation, not a limit.
- Reasoning models (GPT-6, o-series and similar) are often slow. For voice, turn reasoning off or use a non-reasoning model.
- **Run test** shows the time to first token of one turn.
- [Conversation history](/agents/monitor/conversation-history) shows turn latency per session. It separates endpoint time from platform time.
</Tip>

## Limitations

- Single Turn and Tool [agent tests](/agents/test/agent-tests) are not supported. A scripted test run refuses to execute rather than substitute a platform model for yours. Simulation tests run on your endpoint.
- Configuration is API-only for now. A console UI comes later.
- The prompt-level safety guardrails still ride the assembled context, but your model decides whether to honor them.
Loading