Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
125 changes: 125 additions & 0 deletions api-reference/server/services/stt/rime.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
---
title: "Rime Speech-to-Text"
sidebarTitle: "Rime"
description: "Streaming STT with RimeSTTService: each VAD speech segment goes to Rime as its own request, with interim transcripts and one final."
---

## Overview

`RimeSTTService` transcribes speech with Rime's streaming recognition. Rime transcribes one utterance per stream, so the service streams each VAD speech segment as its own request and pushes interim transcripts followed by one final.

<CardGroup cols={2}>
<Card
title="Rime STT API Reference"
icon="code"
href="https://reference-server.pipecat.ai/en/latest/api/pipecat.services.rime.stt.html"
>
Pipecat's API methods for Rime STT integration
</Card>
<Card
title="Example Implementation"
icon="play"
href="https://github.com/pipecat-ai/pipecat/blob/main/examples/transcription/transcription-rime.py"
>
Complete example with Rime STT
</Card>
<Card title="Rime Documentation" icon="book" href="https://docs.rime.ai/">
Official Rime API documentation
</Card>
</CardGroup>

## Installation

```bash
uv add "pipecat-ai[rime]"
```

## Prerequisites

### Rime Account Setup

Before using `RimeSTTService`, you need:

1. **Rime Account**: Sign up at [Rime AI](https://docs.rime.ai/)
2. **API Key**: Generate an API key from your account dashboard

### Required Environment Variables

- `RIME_API_KEY`: Your Rime API key for authentication

### Voice Activity Detection

`RimeSTTService` starts a request on `VADUserStartedSpeakingFrame` and ends it on `VADUserStoppedSpeakingFrame`. Use the user aggregator's VAD analyzer or a `VADProcessor` so these frames reach the service.

## Configuration

### RimeSTTService

```python
from pipecat.services.rime.stt import RimeSTTService
```

<ParamField path="api_key" type="str | None" default="None">
Rime API key. When `None`, the `RIME_API_KEY` environment variable is used.
</ParamField>

<ParamField path="sample_rate" type="int | None" default="None">
Audio sample rate in Hz. When `None`, the pipeline's input sample rate is
used. Rime takes 8000, 16000, 24000, and 48000 Hz; any other rate is resampled
to 16000 Hz before sending.
</ParamField>

<ParamField path="settings" type="RimeSTTService.Settings" default="None">
Runtime-configurable settings. See [Settings](#settings) below.
</ParamField>

<ParamField path="ttfs_p99_latency" type="float | None" default="RIME_TTFS_P99">
P99 latency from speech end to final transcript, in seconds. Override it for
your deployment. See the [STT
benchmark](https://github.com/pipecat-ai/stt-benchmark).
</ParamField>

### Settings

Runtime-configurable settings passed via the `settings` constructor argument using `RimeSTTService.Settings(...)`. These can be updated mid-conversation with `STTUpdateSettingsFrame`, and apply to speech segments that start afterwards. See [Service Settings](/pipecat/fundamentals/service-settings) for details.

| Parameter | Type | Default | Description |
| ----------------- | --------------------------------------------------- | ---------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `model` | `None` | `None` | Must be `None`: Rime selects the recognizer. _(Inherited from base STT settings.)_ |
| `language` | `Language \| str` | `"en"` | Spoken language as a BCP-47 tag, passed to Rime unchanged. See Rime's documentation for supported languages. Must be a nonempty string. _(Inherited from base STT settings.)_ |
| `mode` | `Literal["written", "verbatim"]` | `"written"` | `"written"` formats the transcript as written text. `"verbatim"` transcribes the words as spoken. |
| `context_terms` | `Sequence[str]` | `()` | Recognition hints, such as product names, kept in order. Each term must be nonempty and free of `<\|...\|>` control markers, and the terms joined with `", "` must fit in 1,024 UTF-8 bytes. |
| `output_contract` | `Literal["revised_hypotheses", "append_fragments"]` | `"revised_hypotheses"` | How Rime sends interim results: as replacement hypotheses or as append-only fragments. Both produce cumulative interim transcripts and one final. |

The constructor raises `ValueError` for an invalid value or a non-empty `extra`. An invalid settings update pushes an `ErrorFrame` and leaves the settings unchanged.

## Usage

### Basic Setup

```python
import os

from pipecat.services.rime.stt import RimeSTTService

stt = RimeSTTService(api_key=os.getenv("RIME_API_KEY"))
```

### With Context Terms

```python
stt = RimeSTTService(
api_key=os.getenv("RIME_API_KEY"),
settings=RimeSTTService.Settings(
language="en",
mode="verbatim",
context_terms=["Pipecat", "Rime"],
),
)
```

## Notes

- **Segment length**: A speech segment that runs about 90 to 110 seconds without a pause fails with no transcript.
- **Transcript order**: Finals are pushed in speech order. When the user resumes speaking before an earlier segment's final arrives, only the latest segment's final finalizes the turn.
- **Errors**: A failed request pushes a non-fatal `ErrorFrame` naming the Rime error type and request ID.
1 change: 1 addition & 0 deletions api-reference/server/services/supported-services.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -229,6 +229,7 @@ Speech-to-Text services receive and audio input and output transcriptions.
| [OpenAI](/api-reference/server/services/stt/openai) | `uv add "pipecat-ai[openai]"` | Pipecat |
| [Oruk](/api-reference/server/services/stt/oruk) | `uv add "pipecat-oruk==0.1.0rc1"` | Community |
| [Quickdial](/api-reference/server/services/stt/quickdial) | `uv add pipecat-quickdial` | Community |
| [Rime](/api-reference/server/services/stt/rime) | `uv add "pipecat-ai[rime]"` | Pipecat |
| [Ringg AI](/api-reference/server/services/stt/ringg) | `uv pip install git+https://github.com/Stonkr/pipecat-ringg.git` | Community |
| [RTZR](/api-reference/server/services/stt/rtzr) | `uv add pipecat-rtzr` | Community |
| [Sarvam](/api-reference/server/services/stt/sarvam) | `uv add "pipecat-ai[sarvam]"` | Pipecat |
Expand Down
1 change: 1 addition & 0 deletions docs.json
Original file line number Diff line number Diff line change
Expand Up @@ -451,6 +451,7 @@
"api-reference/server/services/stt/oruk",
"api-reference/server/services/stt/p99lab",
"api-reference/server/services/stt/quickdial",
"api-reference/server/services/stt/rime",
"api-reference/server/services/stt/ringg",
"api-reference/server/services/stt/rtzr",
"api-reference/server/services/stt/sarvam",
Expand Down
126 changes: 126 additions & 0 deletions llms-full.txt
Original file line number Diff line number Diff line change
Expand Up @@ -37666,6 +37666,7 @@ Speech-to-Text services receive and audio input and output transcriptions.
| [OpenAI](/api-reference/server/services/stt/openai) | `uv add "pipecat-ai[openai]"` | Pipecat |
| [Oruk](/api-reference/server/services/stt/oruk) | `uv add "pipecat-oruk==0.1.0rc1"` | Community |
| [Quickdial](/api-reference/server/services/stt/quickdial) | `uv add pipecat-quickdial` | Community |
| [Rime](/api-reference/server/services/stt/rime) | `uv add "pipecat-ai[rime]"` | Pipecat |
| [Ringg AI](/api-reference/server/services/stt/ringg) | `uv pip install git+https://github.com/Stonkr/pipecat-ringg.git` | Community |
| [RTZR](/api-reference/server/services/stt/rtzr) | `uv add pipecat-rtzr` | Community |
| [Sarvam](/api-reference/server/services/stt/sarvam) | `uv add "pipecat-ai[sarvam]"` | Pipecat |
Expand Down Expand Up @@ -51497,6 +51498,131 @@ stt = QuickdialSTTService(
- [pipecat-quickdial on GitHub](https://github.com/samay-ai/pipecat-quickdial) — source, example, and issues
- [Quickdial API docs](https://quickdial.ai/docs)

# Rime Speech-to-Text
Source: https://docs.pipecat.ai/api-reference/server/services/stt/rime.md

Streaming STT with RimeSTTService: each VAD speech segment goes to Rime as its own request, with interim transcripts and one final.

## Overview

`RimeSTTService` transcribes speech with Rime's streaming recognition. Rime transcribes one utterance per stream, so the service streams each VAD speech segment as its own request and pushes interim transcripts followed by one final.

<CardGroup cols={2}>
<Card
title="Rime STT API Reference"
icon="code"
href="https://reference-server.pipecat.ai/en/latest/api/pipecat.services.rime.stt.html"
>
Pipecat's API methods for Rime STT integration
</Card>
<Card
title="Example Implementation"
icon="play"
href="https://github.com/pipecat-ai/pipecat/blob/main/examples/transcription/transcription-rime.py"
>
Complete example with Rime STT
</Card>
<Card title="Rime Documentation" icon="book" href="https://docs.rime.ai/">
Official Rime API documentation
</Card>
</CardGroup>

## Installation

```bash
uv add "pipecat-ai[rime]"
```

## Prerequisites

### Rime Account Setup

Before using `RimeSTTService`, you need:

1. **Rime Account**: Sign up at [Rime AI](https://docs.rime.ai/)
2. **API Key**: Generate an API key from your account dashboard

### Required Environment Variables

- `RIME_API_KEY`: Your Rime API key for authentication

### Voice Activity Detection

`RimeSTTService` starts a request on `VADUserStartedSpeakingFrame` and ends it on `VADUserStoppedSpeakingFrame`. Use the user aggregator's VAD analyzer or a `VADProcessor` so these frames reach the service.

## Configuration

### RimeSTTService

```python
from pipecat.services.rime.stt import RimeSTTService
```

<ParamField path="api_key" type="str | None" default="None">
Rime API key. When `None`, the `RIME_API_KEY` environment variable is used.
</ParamField>

<ParamField path="sample_rate" type="int | None" default="None">
Audio sample rate in Hz. When `None`, the pipeline's input sample rate is
used. Rime takes 8000, 16000, 24000, and 48000 Hz; any other rate is resampled
to 16000 Hz before sending.
</ParamField>

<ParamField path="settings" type="RimeSTTService.Settings" default="None">
Runtime-configurable settings. See [Settings](#settings) below.
</ParamField>

<ParamField path="ttfs_p99_latency" type="float | None" default="RIME_TTFS_P99">
P99 latency from speech end to final transcript, in seconds. Override it for
your deployment. See the [STT
benchmark](https://github.com/pipecat-ai/stt-benchmark).
</ParamField>

### Settings

Runtime-configurable settings passed via the `settings` constructor argument using `RimeSTTService.Settings(...)`. These can be updated mid-conversation with `STTUpdateSettingsFrame`, and apply to speech segments that start afterwards. See [Service Settings](/pipecat/fundamentals/service-settings) for details.

| Parameter | Type | Default | Description |
| ----------------- | --------------------------------------------------- | ---------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `model` | `None` | `None` | Must be `None`: Rime selects the recognizer. _(Inherited from base STT settings.)_ |
| `language` | `Language \| str` | `"en"` | Spoken language as a BCP-47 tag, passed to Rime unchanged. See Rime's documentation for supported languages. Must be a nonempty string. _(Inherited from base STT settings.)_ |
| `mode` | `Literal["written", "verbatim"]` | `"written"` | `"written"` formats the transcript as written text. `"verbatim"` transcribes the words as spoken. |
| `context_terms` | `Sequence[str]` | `()` | Recognition hints, such as product names, kept in order. Each term must be nonempty and free of `<\|...\|>` control markers, and the terms joined with `", "` must fit in 1,024 UTF-8 bytes. |
| `output_contract` | `Literal["revised_hypotheses", "append_fragments"]` | `"revised_hypotheses"` | How Rime sends interim results: as replacement hypotheses or as append-only fragments. Both produce cumulative interim transcripts and one final. |

The constructor raises `ValueError` for an invalid value or a non-empty `extra`. An invalid settings update pushes an `ErrorFrame` and leaves the settings unchanged.

## Usage

### Basic Setup

```python
import os

from pipecat.services.rime.stt import RimeSTTService

stt = RimeSTTService(api_key=os.getenv("RIME_API_KEY"))
```

### With Context Terms

```python
stt = RimeSTTService(
api_key=os.getenv("RIME_API_KEY"),
settings=RimeSTTService.Settings(
language="en",
mode="verbatim",
context_terms=["Pipecat", "Rime"],
),
)
```

## Notes

- **Segment length**: A speech segment that runs about 90 to 110 seconds without a pause fails with no transcript.
- **Transcript order**: Finals are pushed in speech order. When the user resumes speaking before an earlier segment's final arrives, only the latest segment's final finalizes the turn.
- **Errors**: A failed request pushes a non-fatal `ErrorFrame` naming the Rime error type and request ID.

# Ringg AI Speech-to-Text
Source: https://docs.pipecat.ai/api-reference/server/services/stt/ringg.md

Expand Down
1 change: 1 addition & 0 deletions llms.txt
Original file line number Diff line number Diff line change
Expand Up @@ -385,6 +385,7 @@ LLM, transports, serializers), pipelines, frames, workers, and utilities.
- [Oruk Speech-to-Text](https://docs.pipecat.ai/api-reference/server/services/stt/oruk.md): Oruk realtime speech and phrase estimates, plus optional local Orukeet CPU transcription for completed utterances.
- [Audrey STT (p99lab)](https://docs.pipecat.ai/api-reference/server/services/stt/p99lab.md): AudreySTTService streams real-time speech-to-text over a WebSocket to Audrey, p99lab's speech-to-text model for phone calls, meetings and accented speech.
- [Quickdial Speech-to-Text](https://docs.pipecat.ai/api-reference/server/services/stt/quickdial.md): QuickdialSTTService extends SegmentedSTTService to transcribe speech with Quickdial's whisper.cpp-powered STT API.
- [Rime Speech-to-Text](https://docs.pipecat.ai/api-reference/server/services/stt/rime.md): Streaming STT with RimeSTTService: each VAD speech segment goes to Rime as its own request, with interim transcripts and one final.
- [Ringg AI Speech-to-Text](https://docs.pipecat.ai/api-reference/server/services/stt/ringg.md): RinggSTTService streams STT through the official ringglabs Python SDK, which manages the WebSocket connection and event parsing.
- [RTZR Speech-to-Text](https://docs.pipecat.ai/api-reference/server/services/stt/rtzr.md): Streaming Korean STT with RTZRSTTService over RTZR VITO's WebSocket API, with interim results and keyword boosting.
- [Sarvam AI Speech-to-Text](https://docs.pipecat.ai/api-reference/server/services/stt/sarvam.md): SarvamSTTService, SarvamRealtimeSTTService, and SarvamHttpSTTService provide STT for Indian languages over WebSocket or REST.
Expand Down
Loading