The URL Shortener, End to End

Primary source: Hello Interview — “Design a URL Shortener Like Bitly” by Evan King (ex-Meta staff engineer). Interleaved with his video explanation and notable points from the reader comments. This breakdown deliberately targets a junior audience and slows down to explain concepts other breakdowns take for granted.

OrientationThe framework the whole answer hangs on

Before touching the whiteboard, Evan lays out the sequence he follows for every system design interview. Knowing the steps is what lets you build up an unfamiliar system linearly instead of panicking.

Step 1Understanding the problem

Functional requirements — the top 3–4 features

Functional requirements are the features the system must have. Zero in on the few that matter and don't get distracted by bells and whistles. Phrase them as users can… statements.

1. Users can create a short URL from a long URL.
   • Optionally specify a custom alias (e.g. short.ly/evan).
   • Optionally specify an expiration date.
2. Users can access the original URL by visiting the short URL.

Below the line (out of scope): user auth & accounts, and analytics on clicks. They add complexity without being core.

Non-functional requirements — the qualities

These are the “-ilities”: latency, scalability, durability, availability, consistency. Go through them one by one and keep only the ones that are uniquely interesting for this system, phrased as the system should… statements and quantified where possible.

1. Uniqueness of short codes — each code maps to exactly one long URL, so nobody is redirected somewhere unexpected.
2. Low-latency redirects — under ~200 ms.
3. High availability > consistency — reliable ~99.99% of the time; eventual consistency is fine.
4. Scale — support 1B shortened URLs and 100M daily active users.

Deciding availability vs consistency (CAP)

In distributed systems you must accept partition tolerance, so CAP really becomes: do I need strong consistency, or high availability? The test Evan uses: does every read have to see the latest write, or the system breaks?

On the whiteboard, this is all you write:

# Functional
- create short url from original url  (+ optional custom alias, expiration)
- access original url by visiting the short url

# Non-functional
- ensure uniqueness of short codes
- low-latency redirects (< 200ms)
- availability >> consistency  (eventual consistency ok)
- scale to 1B urls and 100M DAU

Steps 2–3Core entities & the API

Start with a broad list of entities — the “tables” you'll persist and exchange — without agonizing over columns yet. Tell your interviewer you'll flesh out the full data model during the high-level design.

✦TipThis system is simple enough that you could write the full data model now. For more complex systems it's usually too early — you haven't thought through the API yet. Announcing the plan (“I'll list entities now, columns later”) keeps you and the interviewer aligned.

The API — usually 1:1 with the features

Walk your functional requirements one by one and define the endpoint each needs. For REST, pick the right verb: POST create, GET read, PUT/PATCH update, DELETE delete.

// 1) Shorten a URL — creating a new mapping, so POST
POST /urls
{
  "long_url": "https://www.example.com/some/very/long/url",
  "custom_alias": "optional_custom_alias",
  "expiration_date": "optional_expiration_date"
}
->  { "short_url": "http://short.ly/abc123" }

// 2) Redirect — reading an existing mapping, so GET
GET /{short_code}
->  HTTP 302 redirect to the original long URL

Step 4High-level design

Go endpoint by endpoint and draw the smallest system that satisfies it. Ignore scale, latency, and uniqueness for now — that's what deep dives are for.

3.1 · Creating a short URL

The flow: the server validates the long URL format, generates a short code (deep dive below), and — if a custom alias was given — checks it doesn't already exist before using it. It stores the row and returns the short URL. For the redirect path we'll make the short code the primary key.

3.2 · Redirecting to the original URL

Crucially, short URLs live on a domain you own (short.ly), so every visit to short.ly/abc123 hits your server. It looks up the code, checks expiration (return 410 Gone if expired), and sends an HTTP redirect the browser follows automatically — the user never notices.

HTTP/1.1 302 Found
Location: https://www.original-long-url.com

Deep dive 1Ensuring short codes are unique

Now we open the black box. Three competing constraints, and it's hard to win all three at once:

Option 1 — Long URL prefix ✗

Take the first N characters of the input URL. Immediately fails uniqueness. Every URL under twitter.com/… or facebook.com/… shares a prefix, so you get a one-to-many mapping and can't know which long URL a short code meant.

Option 2 — Hash function (a great solution)

Feed the (canonicalized) long URL to a hash like SHA-256, base62-encode the output, and slice the first ~6–8 characters. Hashes are deterministic with a strong “waterfall” effect — flip one input bit and the whole output changes — so the sliced result behaves just like a random number.

input_url  = "https://www.example.com/some/very/long/url"
# canonicalize first: lowercase host, strip default ports, normalize slash
canonical_url    = canonicalize(input_url)
hash_code        = hash_function(canonical_url)
short_code_enc   = base62_encode(hash_code)
short_code       = short_code_enc[:8]   # 8 chars -> 62^8 ≈ 218 trillion

Handling collisions: put a UNIQUE constraint on the short-code column, and on insert failure retry (bounded, e.g. 3–5 attempts) with a random salt added to the hash. So the write path becomes generate → check/insert → retry on conflict. An extra read, but not a big deal.

Option 3 — Unique counter + base62 (a great solution)

Why introduce randomness at all? Just hand the first URL code 1, the next 2, and so on — then base62-encode the counter to keep it compact. Collisions become impossible because the value always increases.

From the comments: is generating the same short code for the same long URL desirable?

Generally no. Each short URL is a separate entity that can carry its own expiration date and analytics, and forcing dedup would reveal that a link was already created — a privacy leak. Two different users shortening the same long URL getting different codes is fine, even expected.

Deep dive 2Making redirects fast

The expensive step in a redirect is the database lookup. Without help, finding a code among a billion rows means a full table scan — checking every row. We fix that in layers.

Layer 1 — Add an index

Layer 2 — In-memory cache (Redis / Memcached)

Put a cache between server and database. Check it first; on a hit, return the long URL straight from memory. The speed gap is the whole point:

Memory is ~1,000× faster than SSD and ~100,000× faster than HDD.

Layer 3 — CDN & edge computing

Serve the short-URL domain through a CDN with points of presence worldwide. Cache mappings at the edge, and even run redirect logic there (Cloudflare Workers, Lambda@Edge) so popular codes redirect close to the user and never reach your origin — a request from China resolves in China, not California.

Deep dive 3Scaling to 1B URLs & 100M DAU

The cache already handled most read scaling. Now walk the system left to right and check each component for bottlenecks.

The database — smaller than you'd think

Each row: short code ~8 B, long URL ~100 B, createdAt ~8 B, optional alias ~100 B, expiration ~8 B ≈ ~200 B. Round up to 500 B for metadata.

500 bytes × 1,000,000,000 rows = 500 GB

That fits comfortably on a single modern SSD. Writes are tiny too — maybe ~100k new URLs/day ≈ ~1 write/sec — so almost any database works (Postgres, MySQL, DynamoDB). Pick what you know; default to Postgres. Only shard (e.g. by short code, modulo N) if you ever hit a hardware limit.

Splitting read and write services

Sizing the primary server: 100M DAU × ~1 redirect ÷ ~100k sec/day ≈ 1,000 rps even, ~10–100k rps at peak. A single mid-size instance handles ~1,000 concurrent requests, so we must scale out. Since reads dwarf writes, split into two independently scaled services behind an API Gateway (which routes by endpoint) and a load balancer.

The global counter

Pull the counter into a centralized Redis instance that every write service asks for the next value via atomic INCR. To avoid a network hop per write, use counter batching:

High availability

Autoscaling + horizontal scaling already gets you most of the way. Component by component:

• Cache down? No problem — it's read-through, so it reloads from the DB. Slower briefly, no data loss.
• Global counter down? That is a problem. Run Redis in HA mode (Sentinel / Cluster with automatic failover) and periodically snapshot the count to disk.
• Database down? Keep a replica to fail over to, and snapshot to cheap blob storage (S3) hourly so you can rebuild.

Step 4, evolvedThe final design

CalibrationWhat's expected at each level

ReconstructActive recall

Answer before expanding. These test whether you can rebuild the reasoning, not recite facts.

Why does this system choose availability over consistency? Give the test you'd apply.

Ask: does every read have to see the latest write or the system breaks? For a shortener, no — there's a natural delay before a new link is clicked, and a rare early failure can just show “try again in a minute.” Contrast with ticket booking or banking, where stale reads cause double-bookings or bad trades. So we pick high availability + eventual consistency.

Explain, from scratch, why a randomly generated 6-char code isn't as safe from collisions as it looks.

The Birthday Paradox. Even with 62⁶ ≈ 56B possible codes, generating 1B of them yields ~880k collisions, because collision probability grows with the number of pairs, not just the ratio to the space. Formally the next code collides with probability n/|S|. So you still need a UNIQUE constraint + retry, or a scheme that can't collide.

What breaks when you horizontally scale the write service, and how do you fix it?

An in-memory counter per instance hands out duplicate codes (each starts at 0). Fix: move to a single global counter in Redis using atomic INCR. To cut network hops, batch (grab 1,000 values at a time); losing unused values on crash is fine since we only need uniqueness. For multi-region, assign disjoint ranges.

Walk the redirect path and name every latency optimization in order.

Client → API Gateway → Read Service → Redis cache (hit = O(1) from memory, ~100 ns) → on miss, DB lookup via B-tree/primary-key index (O(log n), avoids full scan) → populate cache → return 302. Optional further layer: CDN/edge cache so popular codes never reach the origin. Layers: index → cache → (edge).

When would 301 beat 302, and what's the hidden danger of 301 with expiration?

301 (permanent, cacheable) wins when you're optimizing for compute cost and don't need per-click visibility — browsers/CDNs handle it and it may never hit your server. Danger: a cached 301 can redirect indefinitely, so you can't reliably stop it at expires_at (proxies may ignore Cache-Control), and you permanently lose analytics for anyone who already cached it. Prefer 302 when expiration or monitoring matter.

How do you argue that a single Postgres instance is enough for storage here?

Row ≈ 500 B; 500 B × 1B rows = ~500 GB, well within one SSD. Writes are ~1/sec. Reads are offloaded to the cache. So no sharding needed — and this is math done because it settles the decision, exactly when Evan says to do it. Shard by short code only if a hardware limit is ever hit.

What's a bijective function doing in this design, and what problem does it solve?

A reversible 1-to-1 map (e.g. Sqids, or a fixed-width block cipher) applied to the sequential counter before/instead of plain base62. It produces a Bitly-looking code that hides the underlying sequence, so competitors can't guess the next code or scrape URLs — while preserving guaranteed uniqueness (no added collisions).

60-second refreshQuick revision