DUCK-E

DUCK-E (Digitally Unified Conversational Knowledge Engine) is a browser-based voice assistant for rubber-duck debugging: you explain a problem out loud and it answers, asks questions, and can look things up. It is a FastAPI service that brokers a WebRTC session between the browser and the OpenAI Realtime API, plus a small set of server-side tools (weather, web search, URL fetch, per-user memory, voice switching). The backend is a custom RealtimeSession class; an earlier AG2/AutoGen dependency was removed and only the ag2.init client message name remains for compatibility.

How it works

Browser (WebRTC) → WebSocket → FastAPI → OpenAI Realtime API
                                              (gpt-realtime-2)

1. Your browser captures audio via the MediaDevices API and opens a WebRTC peer connection.
2. The FastAPI backend requests an ephemeral key from OpenAI — your real API key never touches the browser.
3. Speech is transcribed by OpenAI Whisper-1; the Realtime API handles both understanding and voice response natively.
4. When DUCK-E needs external data (weather, web search), it calls the appropriate tool on the server and folds the result into its reply.

Features

Voice I/O

• Full-duplex voice conversation over WebRTC via the OpenAI Realtime API
• 11 voices (the Realtime API's set) — changeable mid-session via session.update, no reconnect
• Responses can be interrupted mid-sentence; the client finalizes the cut-off transcript
• Optional push-to-talk mode (hold a button or the spacebar) instead of an always-open microphone

Tools

Tool	What it does
get_current_weather	Current conditions via Open-Meteo (free, no key required)
get_weather_forecast	Multi-day forecast via Open-Meteo
web_search	Live web search via OpenAI's gpt-4o-mini + web_search_preview
web_fetch	Fetches and parses a URL (SSRF-protected; blocks private IPs)
save_memory / recall_memories	Stores and retrieves facts about the user
change_voice	Switches voice mid-session

Persistent memory

DUCK-E stores per-user facts with categories, confidence scores, and time decay, and surfaces them at the start of each session. Memory is keyed by user identity via:

• Google OAuth (recommended): Users sign in with Google, memory is keyed to their email
• Reverse proxy headers: Falls back to x-forwarded-user / x-forwarded-email headers if OAuth is not configured

Memory is silently disabled in local dev when neither authentication method is available.

Cost protection

• Per-session spend cap (default: $5)
• Hourly spend cap (default: $50)
• Circuit-breaker threshold (default: $100 — disables new sessions for 30 minutes)
• Maximum session duration (default: 30 minutes)
• Client-side live cost display derived from token counts in response.done events

Security

• Ephemeral key flow — real OPENAI_API_KEY never sent to the browser
• Per-IP rate limiting via slowapi
• SSRF protection on web_fetch (resolves hostnames, rejects private/loopback ranges)
• Input validation via Pydantic models (LocationInput, SearchQuery, FetchUrl)
• CORS origin whitelist
• Security headers middleware (HSTS, CSP, X-Frame-Options, etc.)

Quick start

Docker (fastest)

docker run -d \
  -p 8000:8000 \
  -e OPENAI_API_KEY=sk-... \
  ronaldraygun/duck-e:latest

Open http://localhost:8000 and start talking.

Docker Compose

Create a .env file:

OPENAI_API_KEY=sk-...

Then run:

docker-compose up -d

Local development

pip install -r requirements.txt
# create .env with OPENAI_API_KEY=sk-...
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000

Environment variables

Variable	Required	Default	Description
OPENAI_API_KEY	Yes	—	Must have Realtime API access
REALTIME_MODEL	No	gpt-realtime-2	Override the realtime model
GOOGLE_CLIENT_ID	No	—	Google OAuth client ID (for user authentication)
GOOGLE_CLIENT_SECRET	No	—	Google OAuth client secret
GOOGLE_REDIRECT_URI	No	http://localhost:8000/auth/callback	OAuth callback URL
JWT_SECRET_KEY	No	your-secret-key-here-change-in-production	JWT signing secret
JWT_ALGORITHM	No	HS256	JWT algorithm
JWT_EXPIRATION_MINUTES	No	120	JWT token expiration time
RATE_LIMIT_ENABLED	No	true	Toggle per-IP rate limiting
RATE_LIMIT_WEBSOCKET	No	5/minute	Per-IP WebSocket connection rate
COST_PROTECTION_ENABLED	No	true	Toggle cost protection
COST_PROTECTION_MAX_SESSION_COST_USD	No	5.0	Per-session spend cap
COST_PROTECTION_MAX_TOTAL_COST_PER_HOUR_USD	No	50.0	Hourly spend cap
COST_PROTECTION_CIRCUIT_BREAKER_THRESHOLD_USD	No	100.0	Kill-switch threshold
COST_PROTECTION_MAX_SESSION_DURATION_MINUTES	No	30	Maximum session length
ALLOWED_ORIGINS	No	—	CORS origin whitelist (comma-separated)
GRAFANA_PASSWORD	No	—	For the hardened docker-compose stack

API endpoints

Method	Path	Description
GET	/	Main chat UI
WS	/session	Real-time audio WebSocket
GET	/status	Health check + version
GET	/health/openai	Tests ephemeral key creation
GET	/metrics	Prometheus metrics
GET	/auth/login	Initiate Google OAuth flow
GET	/auth/callback	OAuth callback handler
GET	/auth/config	Check OAuth configuration
GET	/auth/me	Validate JWT and get user info

Voices

DUCK-E ships with 11 voices from the OpenAI Realtime API. Switch any time via the voice selector in the UI or by asking DUCK-E to change its voice mid-conversation:

alloy · ash · ballad · coral · echo · fable · nova · onyx · sage · shimmer · verse

License

MIT — see LICENSE.

---

Part of jedarden.com · Read the write-up: jedarden.com/projects/ducke/

This GitHub repo is a read-only mirror of git.ardenone.com/jedarden/duck-e — issues and PRs are welcome here either way.