diff --git a/agents/build/custom-llm.mdx b/agents/build/custom-llm.mdx index 20ebeee..fcf7d75 100644 --- a/agents/build/custom-llm.mdx +++ b/agents/build/custom-llm.mdx @@ -95,7 +95,24 @@ Requests authenticate with `Authorization: Bearer `. The endpoint ## Point the agent at your endpoint -Set the `llm.custom` section on the agent's [configuration](/agents/build/configuration#configure-through-the-api): +To set it up in the console: + +1. Open the agent's **Configuration** and click the **Model** selector. +2. Choose **+ Custom LLM** at the bottom of the menu. +3. Enter the base URL, the model name and your API key. +4. Click **Run test**. It sends one streaming turn with the agent's prompt and tools, just like a live call. +5. Click **Save and use**. The agent now uses your endpoint. + +**Run test** reports: + +- Whether the endpoint was reached. +- Whether it streamed `delta.content`. +- Whether it accepted the tools. +- The time to first token. + +The saved endpoint appears under **Custom** in the selector. You can edit or remove it there. If you pick a platform model again, the endpoint and the stored key are removed. + +Through the API, set the `llm.custom` section on the agent's [configuration](/agents/build/configuration#configure-through-the-api): ```bash curl --request PATCH "https://api.fish.audio/v1/agent/agents/$AGENT_ID/config" \ @@ -129,11 +146,15 @@ A custom LLM never silently falls back to a platform model, because a platform m - After three consecutive failed generations the agent apologizes, hangs up, and the session ends with `end_reason: llm_endpoint_failure` (in the [sessions API](/agents/monitor/conversation-history#status-and-end-reason), the dashboard, and the `call.ended` webhook's `ended_reason`). Each failure also emits an `llm.endpoint_error` event on the session timeline. -Voice conversations are latency sensitive, so aim for a time-to-first-token under 800 ms. Turn latency is attributed per session in [conversation history](/agents/monitor/conversation-history), which lets you tell endpoint time from platform time. +Voice conversations are latency sensitive: + +- Aim for a time-to-first-token under 800 ms. This is a recommendation, not a limit. +- Reasoning models (GPT-6, o-series and similar) are often slow. For voice, turn reasoning off or use a non-reasoning model. +- **Run test** shows the time to first token of one turn. +- [Conversation history](/agents/monitor/conversation-history) shows turn latency per session. It separates endpoint time from platform time. ## Limitations - Single Turn and Tool [agent tests](/agents/test/agent-tests) are not supported. A scripted test run refuses to execute rather than substitute a platform model for yours. Simulation tests run on your endpoint. -- Configuration is API-only for now. A console UI comes later. - The prompt-level safety guardrails still ride the assembled context, but your model decides whether to honor them.