[Pushkar-spec]

Summary

Replace database polling in the live traffic SSE endpoint with event-driven updates using an in-memory broadcaster when Redis Pub/Sub is unavailable.

Changes Made

• Removed database polling: Eliminated the repeated 1.5-second database queries from /api/traffic/stream.
• Added in-memory broadcasting: Introduced InMemoryTrafficBroadcaster with per-subscriber queues to deliver traffic events to connected clients.
• Added post-commit event publishing: Publish traffic events after successful persistence in both streaming and non-streaming generation paths.
• Improved Redis lifecycle management: Initialize Redis during application startup, handle initialization failures gracefully, and clean up connections during shutdown.
• Preserved Redis Pub/Sub: Retained Redis-based event delivery for multi-instance deployments and added fallback handling for runtime disconnections.
• Preserved the frontend SSE contract: Continue emitting the traffic_event event with the expected traffic payload fields.
• Improved event handling: Added payload normalization, snapshot deduplication, bounded deduplication tracking, and queue management.
• Added automated tests: Cover broadcaster behavior, post-commit publishing, SSE delivery, Redis lifecycle, fallback behavior, and snapshot race conditions.

Testing

• Added tests for in-memory broadcasting, including multiple subscribers and queue overflow.
• Added tests for publishing events only after successful database commits.
• Added SSE tests for initial snapshots, live updates, and the absence of repeated database queries.
• Added Redis lifecycle and fallback tests, including runtime disconnection scenarios.
• Ran the full Python test suite and formatting checks.

Known Limitations

• In-memory queues use a bounded capacity of 100 events per subscriber and drop the oldest queued event when full. Delivery is best-effort for slow clients.
• In-memory broadcasting is process-local. Redis Pub/Sub is required for cross-instance event delivery.
• After a runtime Redis disconnection, an existing SSE connection remains on the in-memory fallback until the client reconnects or the application restarts.

Expected Outcome

Live traffic updates are delivered through event-driven SSE without periodic database polling, while preserving Redis Pub/Sub support and providing a fallback for single-instance deployments without Redis.

[Kaap10]

Reviewed and verified. The in-memory broadcaster cleanly replaces database polling, respects bounded subscriber queues, and preserves the frontend SSE contract. Thank you for your contribution.