Real-time Updates

01The problem, and the two hops

Consider a collaborative editor like Google Docs. When one user types a character, every other viewer needs to see it within milliseconds — not 3–5 seconds later. The naive fix, every user constantly polling “any changes now? now? now?”, would crush your infrastructure with millions of requests every few milliseconds.

The root difficulty: standard HTTP is request-response. Clients ask, servers answer, the connection closes (or at best stays open waiting for another client-initiated request). That model isn’t built to let a server push updates on its own.

02Networking 101

In some sense these are just networking problems. Networks use a layered architecture (the OSI model) where each layer builds on the abstraction below it. Three layers matter most in interviews:

The request lifecycle — and its hidden overhead

Typing a URL kicks off a carefully orchestrated sequence. First DNS resolves the domain to an IP, then:

1. DNS resolution — domain → IP (one round trip; usually cached, but still overhead).
2. TCP handshake (3-way) — SYN → SYN-ACK → ACK establishes the connection.
3. HTTP request — the client sends its actual GET.
4. Server processing — checks a DB, prepares a response.
5. HTTP response — the data comes back.
6. TCP teardown (4-way) — FIN → ACK → FIN → ACK closes the connection.

03L4 vs L7 load balancers

Real systems put multiple servers behind a load balancer. The L4-vs-L7 distinction returns directly when we choose real-time protocols.

Some L7 balancers explicitly support connection-oriented protocols like WebSockets, but generally L4 is better for WebSockets while L7 offers more flexibility for HTTP-based approaches. An L7 balancer doesn’t guarantee the same TCP connection per request — which is exactly the guarantee persistent protocols need.

04Hop 1a · Simple polling — the baseline

The simplest approach: the client makes an HTTP request at a regular interval and the server responds with the current state. For a chat app, you constantly poll “what messages haven’t I received yet?”. It doesn’t technically qualify as real-time, but it’s the right starting point far more often than candidates think.

async function poll() {
  const response = await fetch('/api/updates');
  const data = await response.json();
  processData(data);
}
// Poll every 2 seconds
setInterval(poll, 2000);

Good for: anything not requiring very-low-latency updates, and short update windows (e.g. the LeetCode design). Costs: higher latency (up to interval + processing), limited frequency, more bandwidth, and connection churn with many clients.

05Hop 1b · Long polling — the easy upgrade

// Client-side of long polling
async function longPoll() {
  while (true) {
    try {
      const response = await fetch('/api/updates');
      const data = await response.json();
      processData(data);
    } catch (error) {
      console.error(error);
      await new Promise(r => setTimeout(r, 1000)); // back off on error
    }
  }
}

This cuts the wasteful “no updates” replies. But it has two costs. First, the held-open request is effectively state — and we generally want to minimize state. Second, it introduces extra latency for high-frequency updates, because each update needs the client to “call back” before the next one can be sent.

06Hop 1c · Server-Sent Events — the efficient one-way street

// Client-side
const eventSource = new EventSource('/api/updates');
eventSource.onmessage = (event) => {
  const data = JSON.parse(event.data);
  updateUI(data);
};

// Server-side (Node/Express)
app.get('/api/updates', (req, res) => {
  res.setHeader('Content-Type', 'text/event-stream');
  res.setHeader('Cache-Control', 'no-cache');
  res.setHeader('Connection', 'keep-alive');
  const sendUpdate = (data) => res.write(`data: ${JSON.stringify(data)}\n\n`);
  dataSource.on('update', sendUpdate);
  req.on('close', () => dataSource.off('update', sendUpdate));
});

The key headers: Content-Type: text/event-stream, no caching (it’s real-time, theoretically uncacheable), and connection kept alive. A very popular modern use case is AI chat apps streaming tokens as they’re generated.

07Hop 1d · WebSockets — the full-duplex champion

// Client-side
const ws = new WebSocket('ws://api.example.com/socket');
ws.onmessage = (event) => handleUpdate(JSON.parse(event.data));
ws.onclose = () => setTimeout(connectWebSocket, 1000); // reconnect

08Hop 1e · WebRTC — the peer-to-peer solution

WebRTC enables direct peer-to-peer connections between clients — great for video/audio calls and some data sharing. It sounds odd to discuss on the backend, but setting up a P2P connection involves real backend responsibilities.

How the setup works

1. Signaling server — clients connect here to discover peers and exchange connection info (ICE candidates). It’s lightweight, handles little bandwidth, and is only used in the initial handshake — so it needn’t be heavily latency-optimized. It can be regionalized/partitioned (e.g. never connect North America to Europe) or global.
2. STUN (“Session Traversal Utilities for NAT”) — most clients don’t accept inbound connections for security, sitting behind NAT. STUN uses techniques like “hole punching” to establish a publicly routable address/port, shared via the signaling server.
3. Direct connection — client A connects to client B over those public ports.
4. TURN — fallback when STUN fails (e.g. behind a corporate firewall): relay traffic through a central server. It’s the least desirable mode because all the bandwidth now flows through your servers — the very thing P2P was avoiding.

09Hop 1 · The decision flow

10Hop 2 · Getting updates to the right server

With persistent connections (long-polling, SSE, WebSockets), each client is connected to one server, and that server sends its updates. The problem: to send a message to User C, we must know which server User C is connected to. Three patterns.

Pattern A · Pulling with simple polling

A pull model: the client polls, and the server queries a database for what’s new (e.g. “messages newer than my last one?”). This neatly sidesteps the routing problem — a server only pulls updates for the clients connected to it — and decouples the update source from the receiver via the DB.

Pattern B · Pushing via consistent hashing

Assign each user to a specific server so we always know where they’re connected. Simple modulo hashing (user % N) works when N is fixed but forces almost everyone to reconnect when you scale. Consistent hashing maps servers and users onto a ring; each user connects to the next server clockwise, so scaling only moves the users in affected ring segments.

Pattern C · Pushing via Pub/Sub

A single service collects updates and broadcasts them to interested clients. The Pub/Sub service becomes the main source of state; your persistent connections now terminate at lightweight endpoint servers that just subscribe to topics and forward to clients.

11Putting the two hops together

12Common deep dives

Connection failures & reconnection

Networks are unreliable — mobile drops, WiFi cuts out, servers restart. The challenge is detecting disconnects fast and resuming without data loss. WebSocket breaks don’t always signal cleanly: a client may think it’s connected while the server already cleaned up (“zombie” connections). Use heartbeats to detect them. For recovery, track what each client has received (per-user message queue or sequence numbers) and replay the gap on reconnect — Redis streams are popular here.

The celebrity problem (millions of followers)

When a celebrity posts, millions need the update at once; naive fan-out crashes the system. The fix is strategic caching + hierarchical distribution: cache the update once and distribute through layers — regional servers pull it and push to their local clients — instead of writing to millions of individual feeds.

Message ordering across servers

Two messages sent milliseconds apart can arrive out of order across different paths/servers. Vector clocks or logical timestamps establish ordering relationships. For strict ordering, funnel related messages through a single server or partition — trading some scalability for a simple total order.

14Quick revision