Design: Realtime Chat

What a realtime chat room demands

• Realtime delivery — every message reaches everyone in the room instantly, not on the next poll.
• Many concurrent connections — thousands of users sit idle in rooms, waiting.
• Persistence + history — new joiners need to see what was already said.
• Presence — who's online, who's typing, who just left.
• Multiple server instances — one process can't hold every socket; two users in the same room may land on different servers.

The 10,000-ft view — one message, fanned to everyone

Before the details: here's the whole system in one glance. Follow the numbers left → right.

Gateway + broker + queue

Clients hold a persistent WebSocket to a Gateway; the event loop lets one thread juggle thousands of idle sockets, non-blocking. A sent message publishes onto a Pub/Sub topic per room — every gateway instance subscribed to that room gets it and fans out to its own connected members. A queue durably stores the message and drives async notifications.

broker.publish(
  'room:42', msg);
// every gateway subscribed to
// room:42 gets it → fans out
// to its own connected sockets

One message, delivered everywhere

User A and User B share room 42 but are connected to different gateway instances. Play through the sequence and watch the message fan out through the broker to both.

Follow the numbered messages top-to-bottom — the broker delivers to both gateways (steps 3–4) before either renders the message to its user.

Three primitives, one chat room

• WebSocket gateway + event loop — thousands of idle sockets parked on one non-blocking thread per gateway instance.
• Pub/Sub for cross-instance fan-out — Observer at scale: every gateway subscribes to the rooms it has members in.
• Queue for durable history + async notifications — chat history survives a gateway restart; push notifications fire without blocking the send path.
• Horizontal scaling — add gateway instances freely; the broker, not direct socket references, keeps everyone in sync.