diff --git a/api-reference/server/services/stt/rime.mdx b/api-reference/server/services/stt/rime.mdx new file mode 100644 index 000000000..1c966ec65 --- /dev/null +++ b/api-reference/server/services/stt/rime.mdx @@ -0,0 +1,125 @@ +--- +title: "Rime Speech-to-Text" +sidebarTitle: "Rime" +description: "Streaming STT with RimeSTTService: each VAD speech segment goes to Rime as its own request, with interim transcripts and one final." +--- + +## Overview + +`RimeSTTService` transcribes speech with Rime's streaming recognition. Rime transcribes one utterance per stream, so the service streams each VAD speech segment as its own request and pushes interim transcripts followed by one final. + + + + Pipecat's API methods for Rime STT integration + + + Complete example with Rime STT + + + Official Rime API documentation + + + +## Installation + +```bash +uv add "pipecat-ai[rime]" +``` + +## Prerequisites + +### Rime Account Setup + +Before using `RimeSTTService`, you need: + +1. **Rime Account**: Sign up at [Rime AI](https://docs.rime.ai/) +2. **API Key**: Generate an API key from your account dashboard + +### Required Environment Variables + +- `RIME_API_KEY`: Your Rime API key for authentication + +### Voice Activity Detection + +`RimeSTTService` starts a request on `VADUserStartedSpeakingFrame` and ends it on `VADUserStoppedSpeakingFrame`. Use the user aggregator's VAD analyzer or a `VADProcessor` so these frames reach the service. + +## Configuration + +### RimeSTTService + +```python +from pipecat.services.rime.stt import RimeSTTService +``` + + + Rime API key. When `None`, the `RIME_API_KEY` environment variable is used. + + + + Audio sample rate in Hz. When `None`, the pipeline's input sample rate is + used. Rime takes 8000, 16000, 24000, and 48000 Hz; any other rate is resampled + to 16000 Hz before sending. + + + + Runtime-configurable settings. See [Settings](#settings) below. + + + + P99 latency from speech end to final transcript, in seconds. Override it for + your deployment. See the [STT + benchmark](https://github.com/pipecat-ai/stt-benchmark). + + +### Settings + +Runtime-configurable settings passed via the `settings` constructor argument using `RimeSTTService.Settings(...)`. These can be updated mid-conversation with `STTUpdateSettingsFrame`, and apply to speech segments that start afterwards. See [Service Settings](/pipecat/fundamentals/service-settings) for details. + +| Parameter | Type | Default | Description | +| ----------------- | --------------------------------------------------- | ---------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `model` | `None` | `None` | Must be `None`: Rime selects the recognizer. _(Inherited from base STT settings.)_ | +| `language` | `Language \| str` | `"en"` | Spoken language as a BCP-47 tag, passed to Rime unchanged. See Rime's documentation for supported languages. Must be a nonempty string. _(Inherited from base STT settings.)_ | +| `mode` | `Literal["written", "verbatim"]` | `"written"` | `"written"` formats the transcript as written text. `"verbatim"` transcribes the words as spoken. | +| `context_terms` | `Sequence[str]` | `()` | Recognition hints, such as product names, kept in order. Each term must be nonempty and free of `<\|...\|>` control markers, and the terms joined with `", "` must fit in 1,024 UTF-8 bytes. | +| `output_contract` | `Literal["revised_hypotheses", "append_fragments"]` | `"revised_hypotheses"` | How Rime sends interim results: as replacement hypotheses or as append-only fragments. Both produce cumulative interim transcripts and one final. | + +The constructor raises `ValueError` for an invalid value or a non-empty `extra`. An invalid settings update pushes an `ErrorFrame` and leaves the settings unchanged. + +## Usage + +### Basic Setup + +```python +import os + +from pipecat.services.rime.stt import RimeSTTService + +stt = RimeSTTService(api_key=os.getenv("RIME_API_KEY")) +``` + +### With Context Terms + +```python +stt = RimeSTTService( + api_key=os.getenv("RIME_API_KEY"), + settings=RimeSTTService.Settings( + language="en", + mode="verbatim", + context_terms=["Pipecat", "Rime"], + ), +) +``` + +## Notes + +- **Segment length**: A speech segment that runs about 90 to 110 seconds without a pause fails with no transcript. +- **Transcript order**: Finals are pushed in speech order. When the user resumes speaking before an earlier segment's final arrives, only the latest segment's final finalizes the turn. +- **Errors**: A failed request pushes a non-fatal `ErrorFrame` naming the Rime error type and request ID. diff --git a/api-reference/server/services/supported-services.mdx b/api-reference/server/services/supported-services.mdx index 4b8c95a7b..f7e8e6e9d 100644 --- a/api-reference/server/services/supported-services.mdx +++ b/api-reference/server/services/supported-services.mdx @@ -229,6 +229,7 @@ Speech-to-Text services receive and audio input and output transcriptions. | [OpenAI](/api-reference/server/services/stt/openai) | `uv add "pipecat-ai[openai]"` | Pipecat | | [Oruk](/api-reference/server/services/stt/oruk) | `uv add "pipecat-oruk==0.1.0rc1"` | Community | | [Quickdial](/api-reference/server/services/stt/quickdial) | `uv add pipecat-quickdial` | Community | +| [Rime](/api-reference/server/services/stt/rime) | `uv add "pipecat-ai[rime]"` | Pipecat | | [Ringg AI](/api-reference/server/services/stt/ringg) | `uv pip install git+https://github.com/Stonkr/pipecat-ringg.git` | Community | | [RTZR](/api-reference/server/services/stt/rtzr) | `uv add pipecat-rtzr` | Community | | [Sarvam](/api-reference/server/services/stt/sarvam) | `uv add "pipecat-ai[sarvam]"` | Pipecat | diff --git a/docs.json b/docs.json index dea8227bc..ff629e0e9 100644 --- a/docs.json +++ b/docs.json @@ -451,6 +451,7 @@ "api-reference/server/services/stt/oruk", "api-reference/server/services/stt/p99lab", "api-reference/server/services/stt/quickdial", + "api-reference/server/services/stt/rime", "api-reference/server/services/stt/ringg", "api-reference/server/services/stt/rtzr", "api-reference/server/services/stt/sarvam", diff --git a/llms-full.txt b/llms-full.txt index 0ccc2af5a..8d1f28af9 100644 --- a/llms-full.txt +++ b/llms-full.txt @@ -37666,6 +37666,7 @@ Speech-to-Text services receive and audio input and output transcriptions. | [OpenAI](/api-reference/server/services/stt/openai) | `uv add "pipecat-ai[openai]"` | Pipecat | | [Oruk](/api-reference/server/services/stt/oruk) | `uv add "pipecat-oruk==0.1.0rc1"` | Community | | [Quickdial](/api-reference/server/services/stt/quickdial) | `uv add pipecat-quickdial` | Community | +| [Rime](/api-reference/server/services/stt/rime) | `uv add "pipecat-ai[rime]"` | Pipecat | | [Ringg AI](/api-reference/server/services/stt/ringg) | `uv pip install git+https://github.com/Stonkr/pipecat-ringg.git` | Community | | [RTZR](/api-reference/server/services/stt/rtzr) | `uv add pipecat-rtzr` | Community | | [Sarvam](/api-reference/server/services/stt/sarvam) | `uv add "pipecat-ai[sarvam]"` | Pipecat | @@ -51497,6 +51498,131 @@ stt = QuickdialSTTService( - [pipecat-quickdial on GitHub](https://github.com/samay-ai/pipecat-quickdial) — source, example, and issues - [Quickdial API docs](https://quickdial.ai/docs) +# Rime Speech-to-Text +Source: https://docs.pipecat.ai/api-reference/server/services/stt/rime.md + +Streaming STT with RimeSTTService: each VAD speech segment goes to Rime as its own request, with interim transcripts and one final. + +## Overview + +`RimeSTTService` transcribes speech with Rime's streaming recognition. Rime transcribes one utterance per stream, so the service streams each VAD speech segment as its own request and pushes interim transcripts followed by one final. + + + + Pipecat's API methods for Rime STT integration + + + Complete example with Rime STT + + + Official Rime API documentation + + + +## Installation + +```bash +uv add "pipecat-ai[rime]" +``` + +## Prerequisites + +### Rime Account Setup + +Before using `RimeSTTService`, you need: + +1. **Rime Account**: Sign up at [Rime AI](https://docs.rime.ai/) +2. **API Key**: Generate an API key from your account dashboard + +### Required Environment Variables + +- `RIME_API_KEY`: Your Rime API key for authentication + +### Voice Activity Detection + +`RimeSTTService` starts a request on `VADUserStartedSpeakingFrame` and ends it on `VADUserStoppedSpeakingFrame`. Use the user aggregator's VAD analyzer or a `VADProcessor` so these frames reach the service. + +## Configuration + +### RimeSTTService + +```python +from pipecat.services.rime.stt import RimeSTTService +``` + + + Rime API key. When `None`, the `RIME_API_KEY` environment variable is used. + + + + Audio sample rate in Hz. When `None`, the pipeline's input sample rate is + used. Rime takes 8000, 16000, 24000, and 48000 Hz; any other rate is resampled + to 16000 Hz before sending. + + + + Runtime-configurable settings. See [Settings](#settings) below. + + + + P99 latency from speech end to final transcript, in seconds. Override it for + your deployment. See the [STT + benchmark](https://github.com/pipecat-ai/stt-benchmark). + + +### Settings + +Runtime-configurable settings passed via the `settings` constructor argument using `RimeSTTService.Settings(...)`. These can be updated mid-conversation with `STTUpdateSettingsFrame`, and apply to speech segments that start afterwards. See [Service Settings](/pipecat/fundamentals/service-settings) for details. + +| Parameter | Type | Default | Description | +| ----------------- | --------------------------------------------------- | ---------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `model` | `None` | `None` | Must be `None`: Rime selects the recognizer. _(Inherited from base STT settings.)_ | +| `language` | `Language \| str` | `"en"` | Spoken language as a BCP-47 tag, passed to Rime unchanged. See Rime's documentation for supported languages. Must be a nonempty string. _(Inherited from base STT settings.)_ | +| `mode` | `Literal["written", "verbatim"]` | `"written"` | `"written"` formats the transcript as written text. `"verbatim"` transcribes the words as spoken. | +| `context_terms` | `Sequence[str]` | `()` | Recognition hints, such as product names, kept in order. Each term must be nonempty and free of `<\|...\|>` control markers, and the terms joined with `", "` must fit in 1,024 UTF-8 bytes. | +| `output_contract` | `Literal["revised_hypotheses", "append_fragments"]` | `"revised_hypotheses"` | How Rime sends interim results: as replacement hypotheses or as append-only fragments. Both produce cumulative interim transcripts and one final. | + +The constructor raises `ValueError` for an invalid value or a non-empty `extra`. An invalid settings update pushes an `ErrorFrame` and leaves the settings unchanged. + +## Usage + +### Basic Setup + +```python +import os + +from pipecat.services.rime.stt import RimeSTTService + +stt = RimeSTTService(api_key=os.getenv("RIME_API_KEY")) +``` + +### With Context Terms + +```python +stt = RimeSTTService( + api_key=os.getenv("RIME_API_KEY"), + settings=RimeSTTService.Settings( + language="en", + mode="verbatim", + context_terms=["Pipecat", "Rime"], + ), +) +``` + +## Notes + +- **Segment length**: A speech segment that runs about 90 to 110 seconds without a pause fails with no transcript. +- **Transcript order**: Finals are pushed in speech order. When the user resumes speaking before an earlier segment's final arrives, only the latest segment's final finalizes the turn. +- **Errors**: A failed request pushes a non-fatal `ErrorFrame` naming the Rime error type and request ID. + # Ringg AI Speech-to-Text Source: https://docs.pipecat.ai/api-reference/server/services/stt/ringg.md diff --git a/llms.txt b/llms.txt index b9beb2bec..45c67ca84 100644 --- a/llms.txt +++ b/llms.txt @@ -385,6 +385,7 @@ LLM, transports, serializers), pipelines, frames, workers, and utilities. - [Oruk Speech-to-Text](https://docs.pipecat.ai/api-reference/server/services/stt/oruk.md): Oruk realtime speech and phrase estimates, plus optional local Orukeet CPU transcription for completed utterances. - [Audrey STT (p99lab)](https://docs.pipecat.ai/api-reference/server/services/stt/p99lab.md): AudreySTTService streams real-time speech-to-text over a WebSocket to Audrey, p99lab's speech-to-text model for phone calls, meetings and accented speech. - [Quickdial Speech-to-Text](https://docs.pipecat.ai/api-reference/server/services/stt/quickdial.md): QuickdialSTTService extends SegmentedSTTService to transcribe speech with Quickdial's whisper.cpp-powered STT API. +- [Rime Speech-to-Text](https://docs.pipecat.ai/api-reference/server/services/stt/rime.md): Streaming STT with RimeSTTService: each VAD speech segment goes to Rime as its own request, with interim transcripts and one final. - [Ringg AI Speech-to-Text](https://docs.pipecat.ai/api-reference/server/services/stt/ringg.md): RinggSTTService streams STT through the official ringglabs Python SDK, which manages the WebSocket connection and event parsing. - [RTZR Speech-to-Text](https://docs.pipecat.ai/api-reference/server/services/stt/rtzr.md): Streaming Korean STT with RTZRSTTService over RTZR VITO's WebSocket API, with interim results and keyword boosting. - [Sarvam AI Speech-to-Text](https://docs.pipecat.ai/api-reference/server/services/stt/sarvam.md): SarvamSTTService, SarvamRealtimeSTTService, and SarvamHttpSTTService provide STT for Indian languages over WebSocket or REST.