Voice AI MVP (TypeScript + Gemini Live)

A minimal voice agent using Gemini Live API.

What this app does

• Browser captures microphone audio.
• Backend creates a short-lived Gemini ephemeral token.
• Browser opens a Gemini Live WebSocket session.
• Audio streams to Gemini, model audio streams back.
• Transcript events are shown in the UI.

End-to-end Flow

+------------------+        +----------------------+        +-------------------------+
|   User Browser   |        |  Node/Express Server |        |   Gemini Live API       |
+------------------+        +----------------------+        +-------------------------+
         |                            |                                 |
         | POST /api/gemini/live-token|                                 |
         |--------------------------->|                                 |
         |                            | create ephemeral token          |
         |                            |-------------------------------> |
         |                            |<------------------------------- |
         |                            | token                           |
         |<---------------------------|                                 |
         |    ephemeralToken+config   |                                 |
         |                                                                
         | WebSocket connect with access_token                            |
         |--------------------------------------------------------------->|
         |<---------------------------------------------------------------|
         |             setupComplete + streaming events                   |
         |==================== audio + transcripts ======================>|

Important note about transport

Gemini Live API is implemented over WebSockets (not direct native WebRTC in this app).

Project Structure

• src/server.ts
  • POST /api/gemini/live-token for ephemeral token provisioning.
• src/client.ts
  • WebSocket connect/disconnect.
  • Microphone capture, PCM downsampling (16k), stream send.
  • PCM playback of model audio, transcript display.
• public/index.html
  • Connect / Disconnect / Mute UI.
• .env.example
  • Gemini env vars.

API Endpoint

POST /api/gemini/live-token

Returns:

• ephemeralToken
• model
• voice
• systemPrompt

Environment Variables

Required:

• GEMINI_API_KEY

Optional:

• PORT (default 3000)
• GEMINI_LIVE_MODEL (default gemini-3.1-flash-live-preview)
• GEMINI_VOICE (default Kore)
• GEMINI_SYSTEM_PROMPT
• RATE_LIMIT_WINDOW_MS (default 60000)
• RATE_LIMIT_MAX_REQUESTS (default 10)

Local Setup

cp .env.example .env
# set GEMINI_API_KEY in .env
npm install
npm run dev

Open http://localhost:3000 and click Connect.

Production Build

npm run build
npm start

Deploy

• Build command: npm run build
• Start command: npm start
• Required env: GEMINI_API_KEY

Troubleshooting

• If token creation fails: verify GEMINI_API_KEY.
• If token requests are rejected with 429: increase RATE_LIMIT_MAX_REQUESTS or wait for the window reset.
• If socket connect fails: confirm model name and key permissions.
• If no mic audio: allow browser microphone permission.
• If no sound playback: check browser output device / autoplay policy.

Scope

This is an MVP optimized for quick deployment. It does not yet include auth, billing controls, advanced observability, or telephony adapters.