You’re nearly always designing systems of independent devices that talk over a network. The field is vast, but interviews only lean on a handful of layers and protocols. This is that handful — purpose, mechanism, and when to reach for each.

01Networking 101 & the layered model

At its core, networking is about connecting devices so they can communicate. Networks are built on a layered architecture — the OSI model — which dramatically simplifies life for the application developers sitting on top.

The three layers that matter in interviews

The full stack is fascinating, but three layers come up most. Here’s what each does before we dive in:

Walkthrough — what happens on a simple web request

Type a URL and several layers spring into action. To fetch a page over HTTP on TCP: first DNS converts a human-readable domain like hellointerview.com into an IP like 32.42.52.62. Then a carefully orchestrated sequence sets up a connection, sends the request, gets a response, and tears the connection down.

These handshake details rarely feature directly in a system design interview (they were once a popular “what happens when you type a URL and press enter?” question, less so now in Big Tech). But three observations here we’ll build on:

02Network layer — the IP protocol

The lowest layer we care about is dominated by IP, responsible for routing and addressing. Nodes are usually assigned IPs by a DHCP server when they boot. These addresses are arbitrary — they only mean something because we tell people about them. You can build a private network and assign any IPs you like, but for internet traffic to find your machines you need routable addresses allocated by a RIR.

The key property of public IPs: routing infrastructure knows where they are. Any address starting with 17 (e.g. 17.0.0.0) belongs to Apple — the backbone knows a packet for those addresses goes to Apple’s routers. There’s far more to internet routing, but it won’t matter for our purposes, so we move up the stack.

03Transport layer — TCP, UDP, QUIC

The transport layer establishes end-to-end communication between applications, giving guarantees instead of a jumbled mess of packets. Three protocols are in wide use: TCP, UDP, and QUIC.

For interviews the real choice is usually TCP vs UDP. QUIC is newer — it aims to deliver TCP-like benefits with modernization and performance gains, and it’s gaining ground, but it isn’t yet ubiquitous. Treat it as “a better version of TCP without the same broad adoption.”

UDP — fast but unreliable

UDP is the machine gun of protocols — few features on top of IP, but very fast. “Spray and pray” is the right mental image: connectionless, no delivery guarantee, no ordering, no duplicate protection. If you receive UDP datagrams you can see the source and destination IP + port — the rest is a binary blob.

No setup sounds great, but no delivery guarantee and no ordering kind of suck — so why use it? UDP is perfect where speed matters more than reliability: live video streaming, online gaming, VoIP, DNS lookups. The client is equipped to handle occasional loss.

TCP — reliable but with overhead

TCP is the workhorse of the internet: reliable, ordered, error-checked delivery. It sets up a connection via the three-way handshake (seen above) and maintains it through the session. That connection is a stream — a stateful connection between client and server, and the basis for ordering: two messages in the same stream arrive in the same order. TCP makes recipients acknowledge receipt and retransmits until they do.

TCP is ideal wherever data integrity matters — that is, basically everything UDP is not a good fit for. The reliability comes at a cost in throughput and latency: a lost packet must be retransmitted before things continue, and if the two hosts are far apart that retransmission eats real time.

When to choose each

Most interviewers assume TCP by default — it often needn’t be stated. That’s also the recommendation. You earn points by making a correct case for UDP without bungling the details, so ask yourself whether UDP genuinely fits.

Modern apps often use both: a web-based video-conferencing app might use TCP/HTTP for signaling and auth but UDP/WebRTC for the actual audio/video.

04Application layer protocols

The application layer is where most developers spend their time. These protocols define how applications communicate and are built on the transport protocols above.

HTTP / HTTPS — the web’s foundation

HTTP is the de-facto standard for web data communication: a request–response protocol where clients send requests and servers respond with data. It’s versatile and battle-tested over decades — even RPC applications often ride on it.

You can actually open a TCP connection and type an HTTP request/response by hand with netcat (nc). Requests are simple text: a method (verb) telling your intent, a path, the HTTP version, and headers. Responses carry the data plus a status code and headers.

Methods and status codes are standardized (think enums); headers are flexible key/value pairs. Most interviewers won’t drill into this detail unless you’re designing a RESTful API.

HTTPS adds a security layer (TLS/SSL) to encrypt communications, protecting against eavesdropping and man-in-the-middle attacks. Building a public website? Use HTTPS, without exception — request and response contents are encrypted and safe in transit.

REST — simple and flexible

HTTP builds websites, but system designs usually concern service-to-service APIs. The three main paradigms are REST, GraphQL, and gRPC — and REST is the most common.

If you follow the Delivery Framework, your Core Entities often map directly to your API resources. A simple RESTful API:

GET /users/{id} -> User

PUT /users/{id} -> User
{
  "username": "john.doe",
  "email": "john.doe@example.com"
}

POST /users -> User
{
  "username": "stefan.mai",
  "email": "stefan@hellointerview.com"
}

GET /users/{id}/posts -> [Post]

REST is the default we’d suggest for interviews — well understood, widely usable, and a good baseline for scalable systems. It won’t be the most performant option for very high-throughput services, and JSON is a fairly inefficient serialization format, but most applications aren’t bottlenecked by request serialization. Reach for GraphQL, gRPC, SSE, or WebSockets only when you have a specific need REST can’t meet.

GraphQL — flexible data fetching

Open-sourced by Facebook around 2015, GraphQL lets clients request exactly the data they need. The problem it solves: teams split into frontend and backend, and when the frontend needs a new page it must either (a) cobble together many backend calls, (b) build huge aggregation APIs that are hard to maintain, or (c) write brand-new APIs per page. None are great.

GraphQL lets the frontend describe the shape of data it needs; the backend figures out where to get it and returns it in that shape — and nothing more.

query GetUsersWithProfilesAndGroups($limit: Int = 10, $offset: Int = 0) {
  users(limit: $limit, offset: $offset) {
    id
    username
    profile { id fullName avatar }
    groups {
      id name description
      category { id name icon }
    }
    status { isActive lastActiveAt }
  }
  _metadata { totalCount hasNextPage }
}

gRPC — efficient service communication

gRPC is a high-performance RPC framework from Google (the “g”) built on HTTP/2 and Protocol Buffers. Think of Protocol Buffers like JSON but with a rigid schema (it looks like a C/C++ struct) that enables far more efficient serialization.

message User {
  string id = 1;
  string name = 2;
}

Instead of a chunky JSON object with embedded schema (≈40 bytes)…

{ "id": "123", "name": "John Doe" }

…you get a binary encoding (≈15 bytes) of the same data — skinny tags and variable-length string encoding. Less space, less CPU to parse:

0A 03 31 32 33 12 08 6A 6F 68 6E 20 64 6F 65

message GetUserRequest { string id = 1; }
message GetUserResponse { User user = 1; }

service UserService {
  rpc GetUser (GetUserRequest) returns (GetUserResponse);
}

These compile into client and server stubs consumable by many languages (Java, C++, Node, Ruby, and more). gRPC includes features for microservices at scale — streaming, deadlines, client-side load balancing, rudimentary auth. The headline: it’s a binary protocol that’s faster and more efficient than JSON over HTTP.

Server-Sent Events (SSE) — real-time push

So far, request/response. But many apps need to push data to clients as it happens (stock tickers, notifications). You could poll repeatedly, but then data is delayed by your polling frequency and you pay the overhead of repeatedly opening connections.

{
  "events": [
    { "id": 1, "description": "Event 1" },
    { "id": 2, "description": "Event 2" },
    ...
    { "id": 100, "description": "Event 100" }
  ]
}

data: {"id": 1, "description": "Event 1"}
data: {"id": 2, "description": "Event 2"}
...
data: {"id": 100, "description": "Event 100"}

Where to use it: when clients should get notifications/events as soon as they happen — keeping bidders updated on an auction price, order status over ~15 seconds, or (per the video) streaming AI tokens/responses back to the user as they’re generated, since a full response can take 30 seconds to build.

WebSockets — real-time bidirectional

SSE pushes server→client. When you need real-time bidirectional communication — clients pushing to the server as much as the reverse — you reach for WebSockets: a persistent, TCP-style connection with broad support (including browsers). Unlike request/response, the server can push without being prompted, and clients can too.

Upgrading from HTTP is convenient — you can reuse existing HTTP session info (cookies, headers). WebSockets don’t dictate an application protocol: you get a channel for binary packets and define what’s exchanged (serialized JSON messages are a great option, and let you define your service’s API).

WebRTC — peer-to-peer

The most unique protocol here: WebRTC enables direct peer-to-peer communication between browsers with no intermediary server for the data exchange — great for collaborative editors and especially audio/video calling. It’s also the only application-level protocol here that runs over UDP.

Peer-to-peer is trickier than client-server because most clients don’t accept inbound connections (a security measure) and most users sit behind a NAT device. Clients talk to a central signaling server that tracks who’s available and their connection info; once a client has a peer’s info, it tries to connect directly. To get through NAT, WebRTC defines two mechanisms:

Picking an application protocol

05Load balancing

We’ve covered the protocols. But how do we scale? Two options: bigger servers (vertical scaling) or more servers (horizontal scaling).

Two ways: client-side vs a dedicated appliance

Client-side load balancing

The client itself decides which server to talk to. It queries a service registry/directory listing available servers (or asks the servers themselves), then connects directly, periodically syncing that list. This is very fast and efficient: since the client decides, it skips the extra network hop of routing through a balancer on every request — no additional latency in the middle.

In interviews, client-side balancing works remarkably well for internal microservices (built into gRPC). Many interviewers don’t probe the lines between services, but mention it if asked. For everything else, use a dedicated load balancer.

Dedicated load balancers

Sometimes you don’t want clients refreshing server lists or even knowing multiple servers exist — or you have many uncontrolled clients needing fast updates. A dedicated load balancer is a server or hardware device between clients and backends, deciding where each request goes. It adds a hop per request, but in exchange you get fast server-list updates and fine-grained routing control. Which one you pick depends on the layer it operates at.

Layer 4 load balancers

Operate at the transport layer (TCP/UDP), routing on IPs and ports without inspecting packet contents. The effect is as if you randomly picked a backend and the client had a direct TCP connection to it — the balancer opens a parallel TCP connection to the server and just shuffles packets between the two.

• Maintain persistent TCP connections between client and server.
• Fast and efficient — minimal packet inspection.
• Cannot route on application data.
• Best when raw performance is the priority.

Because a client’s TCP connection through an L4 balancer sticks to the same server for the whole session, L4 is well-suited to protocols needing persistent connections, like WebSockets.

Layer 7 load balancers

Operate at the application layer, understanding protocols like HTTP. They examine request content (URL, headers, cookies) and make smarter routing decisions.

• Terminate incoming connections and create new ones to backends.
• Route based on request content.
• More CPU-intensive (packet inspection).
• More flexibility and features; better for HTTP traffic.

An L7 balancer could send API requests to one server pool and web-page requests to another (like an API Gateway), or pin all of a user’s requests to one server via a cookie. Because it works at the request level, a single client TCP connection carrying multiple HTTP requests can be fanned out across many backends — the underlying TCP connection to your server isn’t really relevant. But you can’t ignore the connection-level reality if you need persistent connections to consistent servers.

Health checks & fault tolerance

Load balancers also monitor backend health — when a server crashes or loses power, the balancer stops routing to it until it recovers. This automatic failover is what makes them essential for high availability. When a server spins up it announces itself; the balancer runs health checks at configurable intervals/protocols and only routes traffic once the server has returned healthy enough times (and stops when it goes unhealthy).

Load balancing algorithms

• Round Robin — sequentially across servers.
• Random — randomly across servers.
• Least Connections — to the server with the fewest active connections.
• Least Response Time — to the fastest-responding server.
• IP Hash — client IP determines the server (useful for session persistence).

Round robin or random is usually fine for stateless apps where no server is more popular than another — and new servers naturally start receiving traffic with no special config. For persistent connections (SSE, WebSockets), use Least Connections to avoid one server gradually accumulating all the long-lived connections (some last minutes, some hours).

06Common deep dives & failure modes

We have protocols, load balancing, persistent connections, and high availability. Beyond those, a few practical considerations are ripe for deep-dive probing — and can make or break a design.

Regionalization & latency

Global services distribute servers around the world. A common pattern: multiple data centers in one region (Amazon calls these availability zones) so a pipe breakage in one building doesn’t take down the service, then replicate that across cities worldwide. Great for humanity — but it introduces networking challenges.

Content Delivery Networks (CDNs)

The most common latency-reduction strategy. CDNs are networks of servers strategically placed around the world — often hundreds or thousands of cities. These edge locations act like gateways: you connect to the nearest one. If that edge server can answer your request, you get lightning-fast responses — the data is just up the road.

This works because of caching: data that doesn’t change often can be cached at the edge and returned from there — especially effective for static content like images and videos. In interviews you’ll see CDNs when data is very cacheable and queried globally (e.g. caching Facebook search results) — minimizing latency and reducing backend load. When data can’t be cached, the edge node forwards to the origin server, acting as a proxy.

Regional partitioning

When you have many users in one region, partition data by region so each region only holds data relevant to it.

Handling failures & fault modes

Beyond server failures (crashes, solar flares flipping bits, cut power), we face network failures: cut cables, failed routers, dropped packets. Robust design plans for all of these.

Timeouts, retries & backoff

The most elementary hygiene. If a request is expected to take a certain time, set a timeout and give up (and retry) if it exceeds it. The worst scenario is making a request, getting an “in progress” acknowledgement, then never hearing back — the client waits forever. But a timeout that’s too short is also bad: a 3-second timeout when the server needs 4 seconds means you fail no matter what. Timeouts must be long enough to let servers respond, short enough to still serve their purpose.

Retries handle transient failures — a temporarily slow server often succeeds on retry. But a naive “retry every 5 seconds” is bad for two reasons: the server probably hasn’t changed, and requests bunch up.

Idempotency

Retries are great — except when they have side effects. Retrying a “charge $10” request could charge the user $20 (or $2,000). The fix: make APIs idempotent — callable multiple times with the same result. Reads are naturally idempotent (an HTTP GET doesn’t change state, even if the content it returns changes). Writes are harder.

Circuit breakers & cascading failures

Senior/staff candidates are often asked “what happens when this service goes down?” Sometimes the answer is simple (“retry until it’s back”) — but retries can create new problems. If a database goes down cold and must boot one instance at a time, a firehose of retries and angry users can pin an instance so it never starts. You can’t bring up the first instance, so you can’t recover the whole database — a thundering herd.

Enter circuit breakers — inspired by the electrical kind. They protect the system when calls to a dependency fail repeatedly:

1. The breaker monitors failures when calling an external service.
2. When failures exceed a threshold, the circuit trips to open.
3. While open, requests fail immediately without attempting the real call.
4. After a timeout, it moves to half-open.
5. A test request determines whether to close the circuit or keep it open.

Good places to apply them: external/third-party API calls, database connections and queries, service-to-service calls in microservices, resource-intensive operations that might time out — any network call that could fail or become slow. They’re a strong answer when an interviewer deep-dives on reliability, failure modes, or disaster recovery.

07Wrapping up

Networking is the foundation connecting all components in a distributed system. The field is vast, but four areas prepare you for most interviews:

1. Understand the basics — IP addressing, DNS, the TCP/IP model.
2. Know your protocols — TCP vs UDP, HTTP/HTTPS, WebSockets, gRPC.
3. Master load balancing — client-side balancing and dedicated load balancers.
4. Plan for practical realities — regionalization and patterns for handling failures.

Networking decisions touch every part of a system — latency, throughput, reliability, security. In interviews, justify your choices against the specific requirements. These defaults are sensible, but most problems have no single right answer; the interviewer wants to see how you reason through trade-offs.

↻Active recall

Answer from memory first, then reveal. These test reconstruction, not trivia.

≡Quick revision

A fast memory refresh — not a replacement for the main notes above.

Study notes built from the Hello Interview guide “Networking Essentials for System Design Interviews” and its companion video walkthrough. Where the video added intuition beyond the written guide, it’s marked as an Instructor insight. One minor mismatch to note: the written guide gives Apple’s IP block as 17.0.0.0, while the video says 18 — the guide is the primary source here.