Serving traffic
The socket only carries the live edge
Own each connection on one gateway, bound its backlog, and recover missed events from durable history.
An independent note, with worked examples. Watch the source lessons.
A comment has a durable journey and a live journey
A viewer posts a comment during video 42. Other viewers should see it quickly, but a disconnected viewer should still find it later. Those are different promises. First commit the comment to an authoritative store with a stable event ID. Then distribute a live notification that says new data is available.
Arpit Bhayani’s account of Trello’s broker migration follows board changes from an API and database through RabbitMQ, then Kafka, to WebSocket servers that know which local users subscribed to each board. It shows why broker topology and connection routing evolve separately even though they meet on the live delivery path. Hello Interview’s auction design cleanly separates committed bid state from an SSE and Pub/Sub path that updates connected viewers. The same separation works for comments, prices, scores and notifications.
The gateway that accepted the socket owns it
A load balancer sends each long-lived SSE or WebSocket connection to one real-time gateway. That process holds the actual socket and an in-memory map from stream IDs to local connections. Another service cannot later send to a stored client IP; it must route the event to the gateway that currently owns the connection.
Gateways can subscribe to the streams represented by their local viewers, or a routing layer can maintain gateway membership per stream. A popular stream must span many gateways. Hashing every viewer for video 42 onto one machine only turns popularity into a hot server.
Hello Interview’s networking survey frames SSE as server-to-client streaming and WebSockets as bidirectional. Choose from the interaction: live comments can arrive over SSE while writes use ordinary HTTP; collaborative editing may need a WebSocket. Both choices still require connection limits, heartbeats, proxy behavior and reconnect logic.
Follow event 7314 through the system
In our worked example, the comment service commits {stream: 42, event: 7314, comment: c918}. A transactional outbox record is committed beside it. A relay publishes that record after commit, closing the crash gap between “saved” and “announced.” Gateways with viewers of stream 42 receive the envelope and enqueue it only for their local connections.
commit comment c918 and outbox event 7314
relay publishes stream 42 / event 7314
gateway A -> connections a1, a2
gateway C -> connection c7The envelope carries an ID and enough routing data, not the sole copy of history. Consumers tolerate a duplicate publish by deduplicating on the event ID. If ordering matters, one authoritative sequence for stream 42 and one ordered partition make the contract explicit; a wall-clock timestamp from each client does not.
A slow viewer cannot own unbounded memory
Connection a2 stops reading while a1 remains fast. If the gateway waits for a2 before sending to a1, one mobile connection delays the stream. If it keeps appending to a2’s private queue, memory grows until the gateway fails and drops everyone.
Give every connection a bounded outbound buffer and a policy. Presence updates can often be coalesced to the newest state. Durable comments cannot be silently discarded, so a connection that crosses its high-water mark can be closed and allowed to reconnect with its last processed event ID. Apply per-stream and per-tenant quotas as well, because one celebrity stream can fill every connection buffer at once. Backpressure here protects shared gateways; durable history preserves what the disconnected viewer has not yet read.
Reconnect from history, not from Pub/Sub
Suppose a2 last processed event 7311, then its gateway crashed while 7312–7314 were published. On reconnect it sends that cursor. The new gateway establishes a live subscription, obtains a durable high-water mark, backfills events after 7311 through that mark, and buffers newer live events until the backfill finishes. That mark must identify a fully committed, readable prefix, not merely the largest ID allocated; transactions that commit out of order can otherwise leave a hidden gap below it. The boundary event may arrive twice, so the client or gateway deduplicates by ID.
Redis Pub/Sub can be a useful low-latency signal between healthy processes, but it is not durable history. A subscriber that is absent or disconnected misses publications. Browser connectivity does not prove broker continuity: if a gateway loses its subscription, observes a sequence gap, or rejoins after a rebalance, it should catch up from durable history even when the browser socket never closed. The auction lesson makes the essential recovery boundary visible: authoritative state survives separately from the live channel. Redis Streams, Kafka or another retained log can add broker replay, but retention, consumer state and recovery still need a defined contract.
Test three failures against event 7314: the relay crashes after commit, gateway A dies after sending only to a1, and a2 reconnects during backfill. If every case names the durable cursor and the component that resumes work, the design has a recovery story rather than only a fast socket.