diff --git a/api-reference/asyncapi-asr-realtime.yml b/api-reference/asyncapi-asr-realtime.yml index e7f708c..f060165 100644 --- a/api-reference/asyncapi-asr-realtime.yml +++ b/api-reference/asyncapi-asr-realtime.yml @@ -93,7 +93,7 @@ channels: enum: - transcribe-stream-1 default: transcribe-stream-1 - description: Realtime speech-to-text model. Optional; `transcribe-stream-1` is the default and the only realtime model. The legacy name `transcribe-1-pro-realtime` reaches the same model. + description: Realtime speech-to-text model. Optional; `transcribe-stream-1` (Preview) is the default and the only realtime model. The legacy name `transcribe-1-pro-realtime` reaches the same model. description: | Realtime speech-to-text channel. Clients stream audio and receive transcripts turn by turn, with turn detection, optional live drafts and @@ -101,7 +101,7 @@ channels: ## Connection Headers - `Authorization: Bearer ` - Required for authentication (see security section) - - `model: transcribe-stream-1` - Optional; the default + - `model: transcribe-stream-1` (Preview) - Optional; the default Every message in both directions is one MessagePack map in a binary frame, with a `type` field. Send `config` first, wait for diff --git a/api-reference/endpoint/websocket/asr-realtime.mdx b/api-reference/endpoint/websocket/asr-realtime.mdx index 4f049ee..01e3c4b 100644 --- a/api-reference/endpoint/websocket/asr-realtime.mdx +++ b/api-reference/endpoint/websocket/asr-realtime.mdx @@ -4,6 +4,7 @@ title: "WebSocket Realtime Speech to Text" description: "Stream audio over a WebSocket and receive transcripts turn by turn, with turn detection, live drafts and conversation context" icon: "webhook" iconType: "solid" +tag: "Preview" --- @@ -13,6 +14,8 @@ iconType: "solid" [Realtime Speech to Text](/features/speech-to-text/realtime). +`transcribe-stream-1` is in preview. + ## Before you start You need a [Fish Audio API key](/developer-guide/getting-started/api-key) and a WebSocket client that can set request headers. The API key travels in a request header, so connect from your server, not from a browser; to capture audio in a browser, relay it through your server as in [Transcribe a live audio stream](/developer-guide/sdk-guide/cookbook/realtime-transcription#capture-from-a-browser-microphone). The examples on these pages use Python with `websockets` and `msgpack`, and Node.js with `ws` and `@msgpack/msgpack`. @@ -30,7 +33,7 @@ model: transcribe-stream-1 | Header | Required | Description | | --------------- | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------- | | `Authorization` | Yes | `Bearer `. Write the scheme exactly as `Bearer`, followed by a space and the key. | -| `model` | No | `transcribe-stream-1`, the default and the only realtime model. The legacy name `transcribe-1-pro-realtime` reaches the same model. | +| `model` | No | `transcribe-stream-1` (Preview), the default and the only realtime model. The legacy name `transcribe-1-pro-realtime` reaches the same model. | Each open connection counts against the same concurrency limit as your other API requests, from the moment it opens until it closes, even while you send no audio. See [Concurrent request limits](/developer-guide/models-pricing/pricing-and-rate-limits#concurrent-request-limits). A session cannot be resumed: a new connection is a new session. diff --git a/developer-guide/models-pricing/models-overview.mdx b/developer-guide/models-pricing/models-overview.mdx index 746a748..1ece829 100644 --- a/developer-guide/models-pricing/models-overview.mdx +++ b/developer-guide/models-pricing/models-overview.mdx @@ -77,7 +77,7 @@ We recommend `transcribe-1-pro` for transcribing recordings. Both file models us | -------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `transcribe-1-pro` (recommended) | Anything from short clips to long recordings (up to 60 minutes), including multi-speaker conversations. Speaker markers in `text`, structured `speaker_turns` with start and end times, speaker-count hints, and inline emotion and vocal-event cues such as `[高兴]` (happy) and `[laughter]` (optional via `tag_audio_events`). Supports optional timestamps. | | `transcribe-1` | General transcription of short recordings, with optional timestamps. Serves requests whose `model` header is missing or not an exact match. | -| `transcribe-stream-1` | Live audio streamed over the [realtime WebSocket](/features/speech-to-text/realtime), transcribed turn by turn while people speak. Turn detection for voice agents (eager end of turn, backchannel detection and background awareness), live drafts, conversation context, speaker tags and sound tags. Sessions up to 60 minutes. | +| `transcribe-stream-1` (Preview) | Live audio streamed over the [realtime WebSocket](/features/speech-to-text/realtime), transcribed turn by turn while people speak. Turn detection for voice agents (eager end of turn, backchannel detection and background awareness), live drafts, conversation context, speaker tags and sound tags. Sessions up to 60 minutes. | Pro labels speakers with inline `<|speaker:N|>` markers in `text`, alongside emotion cues. A label identifies the same speaker within one response only. When you request timestamps (`ignore_timestamps=false`), Pro also returns `speaker_turns`, a structured list of turns with each turn's speaker, text, and start and end times, unless you send `diarize=false`. Timestamped `segments` are word-level, contain spoken words without these markers, and do not expose per-segment speaker IDs. See the [multi-speaker response example](/features/speech-to-text#multi-speaker-conversations) and [Speaker turns](/features/speech-to-text#speaker-turns). diff --git a/developer-guide/models-pricing/pricing-and-rate-limits.mdx b/developer-guide/models-pricing/pricing-and-rate-limits.mdx index f12802b..3700069 100644 --- a/developer-guide/models-pricing/pricing-and-rate-limits.mdx +++ b/developer-guide/models-pricing/pricing-and-rate-limits.mdx @@ -39,11 +39,11 @@ the same price. See [Migrate from S1](/developer-guide/models-pricing/migrate-fr ### Automatic Speech Recognition (ASR) Models -| Model Name | Price (USD) | -| ------------------------------------ | -------------------- | -| `transcribe-1-pro` (recommended) | $0.36 / audio hour | -| `transcribe-1` | $0.36 / audio hour | -| `transcribe-stream-1` (Realtime ASR) | $0.36 / session hour | +| Model Name | Price (USD) | +| --------------------------------------------- | -------------------- | +| `transcribe-1-pro` (recommended) | $0.36 / audio hour | +| `transcribe-1` | $0.36 / audio hour | +| `transcribe-stream-1` (Realtime ASR, Preview) | $0.36 / session hour | For recordings, we recommend `transcribe-1-pro`, which handles multi-speaker conversations (speaker markers and speaker turns), long recordings, and inline emotion cues. Both file models cost the same and use the [Speech to Text API](/api-reference/endpoint/openapi-v1/speech-to-text). Send the `model: transcribe-1-pro` HTTP header to select Pro; a request whose `model` header is missing or not an exact match is served and billed as `transcribe-1`. See the [model comparison](/features/speech-to-text#choose-an-asr-model). diff --git a/developer-guide/sdk-guide/cookbook/realtime-transcription.mdx b/developer-guide/sdk-guide/cookbook/realtime-transcription.mdx index 6d4dbca..eb7c010 100644 --- a/developer-guide/sdk-guide/cookbook/realtime-transcription.mdx +++ b/developer-guide/sdk-guide/cookbook/realtime-transcription.mdx @@ -2,10 +2,13 @@ title: "Transcribe a live audio stream in realtime" description: "Stream audio to transcribe-stream-1 over a WebSocket and print drafts, turn events and final transcripts as they arrive" icon: "tower-broadcast" +tag: "Preview" --- import Prerequisites from "/snippets/prerequisites.mdx"; +`transcribe-stream-1` is in preview. + ## Prerequisites diff --git a/developer-guide/sdk-guide/cookbook/realtime-voice-agent.mdx b/developer-guide/sdk-guide/cookbook/realtime-voice-agent.mdx index 06a8aec..7809082 100644 --- a/developer-guide/sdk-guide/cookbook/realtime-voice-agent.mdx +++ b/developer-guide/sdk-guide/cookbook/realtime-voice-agent.mdx @@ -2,6 +2,7 @@ title: "Build a realtime voice agent: streaming speech in, reply, speech out" description: "Use realtime speech to text to know when the user has finished, start replies early, keep talking through backchannels and stop on interruptions" icon: "headset" +tag: "Preview" --- import Prerequisites from "/snippets/prerequisites.mdx"; @@ -12,6 +13,8 @@ import Prerequisites from "/snippets/prerequisites.mdx"; [Agents](/agents/overview). +`transcribe-stream-1` is in preview. + ## Prerequisites diff --git a/features/speech-to-text/realtime.mdx b/features/speech-to-text/realtime.mdx index b9e0c73..683058d 100644 --- a/features/speech-to-text/realtime.mdx +++ b/features/speech-to-text/realtime.mdx @@ -3,10 +3,13 @@ title: "Realtime Speech to Text" sidebarTitle: "Realtime" description: "Transcribe live audio turn by turn with transcribe-stream-1: turn detection that knows when the user has finished, backchannel and background awareness, and conversation context" icon: "microphone-lines" +tag: "Preview" --- Stream live audio over a WebSocket and get the transcript turn by turn while people speak. `transcribe-stream-1` does more than transcribe: it decides when the user has finished their turn, usually tells the user talking to your app apart from background sound, and notices when a "mm-hm" is only an acknowledgement. This helps your voice agent decide when to answer, when to keep talking and when to stop. Connect to `wss://api.fish.audio/v1/asr/realtime`, send audio as you capture it, and read the events the server sends back. +`transcribe-stream-1` is in preview. +