System Design · Common Problem Breakdown

Design a Photo-Sharing App Like Instagram

Two hard problems hiding inside a "medium" question: generating a low-latency feed for 500M DAU, and delivering large photos and videos instantly worldwide. Everything else is scaffolding around those two.

Source: Hello Interview · Evan King Scale: 500M DAU · 100M posts/day Asked at: Meta + all FAANG Patterns: Scaling Reads · Large Blobs · Long-Running Tasks

How to use these notes

This is the written breakdown as the primary source, with the instructor's spoken commentary from the video woven in as Instructor Insight callouts right next to the concept they clarify. Highlight colors are consistent throughout and work in grayscale/print. End with 15 recall questions and a one-page revision sheet.

Understanding the Problem

What is Instagram?

A social media platform focused on visual content — users share photos and videos with their followers. Designing it is one of the most common interview questions not just at Meta but across all FAANG and FAANG-adjacent companies. It overlaps heavily with FB News Feed (the feed) and Dropbox (large blobs), but its popularity earns it a standalone breakdown.

Functional Requirements

The three "users should be able to…" statements the design commits to:

Create posts featuring photos, videos, and a simple caption.
Follow other users.
See a chronological feed of posts from users they follow.

Explicitly out of scope: likes & comments, search (users/hashtags/locations), stories (ephemeral content), and going live (real-time streaming).

Instructor Insight — keep scope tight on purpose

"Commenting and liking would be the big ones we define as out of scope. In some interviews you get to it, but I find it's really uncommon — typically that would blow up the scope quite significantly and you just don't have enough time." The three requirements above are the near-universal ask when someone says "design Instagram."

Non-Functional Requirements

Ask for scale before you write these

Scale directly shapes the design, so confirm it first. Here: 500M DAU, 100M posts/day. The goal of non-functional requirements is not to list every generic quality — it's to name the ones that make this system uniquely challenging.

Highly available, prioritizing availability of photos/videos over consistency — eventual consistency is fine, up to ~2 minutes.
Low-latency feed: < 500ms end-to-end for feed requests.
Low-latency media: photos and videos render near-instantly.
Scalable to 500M DAU while still meeting the latency targets.

Out of scope: security/authz/encryption, strict fault tolerance / no-data-loss guarantees, and analytics.

Instructor Insight — why availability wins the CAP trade

"A user posts in Germany, and I in America load my feed and don't see their post for a couple seconds to minutes. No big deal — that's not the end of the world." Because a slightly stale feed is acceptable, we prioritize availability over consistency. The feed should still load even if it's a little out of date.

Latency mental model

< 200ms feels instant / real-time to a human. 500ms (half a second) is still quick — no one is upset. Once you cross 1 second+ it feels noticeably laggy. That's why 500ms is a defensible target for both feed loading and media rendering.

Section takeaway

Three functional reqs: post · follow · chronological feed. Likes/comments/stories/live are out.
Ask scale first (500M DAU). It justifies every later decision.
Availability > consistency (eventual, ~2 min OK); feed < 500ms; media < 500ms; scale to 500M.

Defining the Core Entities

Start high-level; a short list is enough. You can flesh out the full schema later in the high-level design. Talk through them with your interviewer to stay on the same page.

The four nouns

User — username, profile details (kept minimal).
Post — a reference to the media file, a caption, and the author. One Post entity handles both photos and videos.
Media — the actual bytes of the photo/video. Lives in S3.
Follow — the uni-directional relationship: follower → followee.

Instructor Insight — split Post from Media early

"This is me realizing proactively that Post and Media are going to need to be normalized — separated from one another. Maybe you find this later and come back, but I know right away these two things are stored and exchanged separately." Core entities are the nouns your API exchanges and your DB stores — often one-to-one with tables.

Follow is directional

"I follow @leomessi, but he (sadly) doesn't follow me back." So I see his posts in my feed; he won't see mine. A single Followers table with (followerId, followedId) models this — one new row per follow.

API / System Interface

One endpoint per functional requirement. Start simple; add detail as you go.

Create a post

POST /posts -> postId
{
  "media": {photo or video bytes},
  "caption": "My cool photo!"
}

The actual media upload is handled separately via a pre-signed URL (details in the media deep dive). For now this just creates the post metadata and returns a postId the client uses later.

Follow a user

POST /follows
{
  "followedId": "123"
}

Get the follower ID from the token

The follower's ID comes from the auth token (JWT/session), not the request body — more secure and follows least privilege. Including it in the body is fine too; you'd just compare it to the session before following.

Fetch the feed

GET /feed?cursor={cursor}&limit={page_size} -> Post[]

Cursor pagination plus a limit for page size. Returns an array of posts and a next_cursor for the next page. The shape of each post is left vague on purpose — filled in later.

High-Level Design

Work through the functional requirements one by one, starting with the naive version and improving it in the deep dives.

Both Post Service and Follow Service write to the same DB here — a deliberate, valid simplification (see the shared-DB note below).

1) Create posts (photos, videos, caption)

Assume for now that media is small enough for a single request. The components:

Clients — web/app; all requests route through the API Gateway.
API Gateway — entry point; routing plus cross-cutting concerns (auth, rate limiting, logging).
Post Service — receives media + caption, writes metadata to the DB and bytes to the blob store, returns postId.
Posts DB — who created it, when, caption, link to media in the blob store.
Blob Store (S3) — the actual media bytes.

2) Follow other users

Insert one row (followerId, followedId) per POST /follows. A dedicated Follow Service is split out from posting because follows happen far less often than posting/viewing — letting each service scale to its own needs.

Extra Context — "two services, one database?"

This draws a lot of comments. The purist view is each service owns its data exclusively; in practice that's "more academic than practical." Many production systems at FAANG have multiple services writing to one database.

Shared database	Separate DB per service
+ Simpler cross-domain transactions	+ Stronger isolation / encapsulation
+ Easier referential integrity	+ Independent scaling & tech choice
− Tighter coupling; noisy-neighbor risk	− Needs distributed txns / eventual consistency
− Schema changes need coordination	− More ops overhead & cost

Many orgs start shared and split out later as needs arise. For this design, sharing is perfectly valid — the domains are related and the coupling is natural.

3) Chronological feed (the naive version)

Simplest approach — fan-out-on-read:

Get followees: query the Follow table for everyone the user follows.
Get posts: for each followed user, query the Post table for recent posts.
Merge & sort chronologically (by timestamp or postId).
Return the sorted, paginated posts.

The Post Service handles this for now (no separate service) — a complexity-vs-cost trade with no single right answer.

Indexes to avoid full-table scans

Follower table: partition key = followerId, sort key = followedId → efficiently list everyone a user follows.
Post table: partition key = userId, sort key = composite (createdAt, postId) → chronological order + uniqueness.

DB choice — and a fun fact

Given the scale, limited relationships, and acceptable eventual consistency, the breakdown picks DynamoDB — though PostgreSQL or most databases would be equally valid if you can justify it. Fun fact: real Instagram uses PostgreSQL as its main post-metadata store. So much for "SQL can't scale to 500M DAU."

The problem we just created

"What if I'm following thousands of people? This will be crazy slow, even with indexes." Exactly right — fan-out-on-read doesn't scale to 500M DAU. That's the first deep dive.

Deep Dive · Low-Latency Feed (< 500ms)

Pick up exactly where the high-level design left off. Why doesn't fan-out-on-read scale?

Three problems with fan-out-on-read

A user following 1,000+ accounts needs 1,000+ Post-table queries. DynamoDB batches cap at 100 items / 16MB, so that's ≥10 batch ops — parallelizable, but still:

Read amplification — every refresh spawns many reads; with 500M DAU refreshing repeatedly this gets expensive fast.
Repeated work — users who follow the same accounts re-query the same posts; popular posts get fetched millions of times.
Unpredictable performance — latency swings with how many accounts you follow and how active they are. Consistent p99 becomes nearly impossible.

Back-of-envelope: why 500ms breaks

Each refresh ≈ 10,000 posts to process (1,000 accounts × 10 posts/day).
500M DAU × 5 refreshes/day ≈ 2.5 billion feed generations/day.
Peak (evenings, big events): 150,000+ feed requests/second.

To hold 500ms under that load you'd either massively over-provision (prohibitively expensive) or accept higher latency at peak (violates the requirement). The root cause: fan-out-on-read postpones all the work to read time — exactly when the user is waiting.

Pattern: Scaling Reads

Instagram is the archetypal scaling-reads system — users read hundreds of posts but write only occasionally. That extreme read:write ratio motivates sharding posts by user ID, vertical partitioning by data type, and tiered storage for old content.

Bad

Simple caching on top of fan-out-on-read

Put Redis in front of the Posts table. Before querying, check the cache; on miss, query the DB and store the result. Key by user + cursor: feed:{user_id}:{cursor}; value = list of postIds.

Why it's not enough

The cache would have to be enormous to hit a meaningful ratio at this scale, and you still do expensive fan-out reads to aggregate posts on every miss. It treats the symptom, not the fan-out-on-read root cause.

Good

Precompute feeds — fan-out-on-write

Flip the timing: build the feed when a user posts, not when a user reads. On a new post, find all followers and prepend the new postId to each of their precomputed feeds.

The GSI that makes it work

Add a Global Secondary Index on the Follows table with followedId as partition key and followerId as sort key. Now "who follows the user who just posted?" is a single fast query.

Redis feed data model

Key: feed:{user_id}
Type: Sorted Set (ZSET)
Members: postId
Scores: timestamp (or postId, if chronologically sortable)

Reading a feed is now one fast ZSET read of the top N. But you still need to hydrate postIds into full post data. Three options:

Hydration option	Trade-off
1 · Store only postIds; fetch metadata from DynamoDB per request	Simple, but an extra DB query on every feed read
2 · Cache full post metadata in Redis	Faster, but more memory + consistency challenges
3 · Hybrid: ZSET of postIds + short-lived HASH metadata cache	Best balance. Try Redis first; batch-fetch misses via `BatchGetItem`; backfill cache

Option 3 wins: tune the metadata cache TTL to your memory/consistency needs. If a post's caption changes, invalidate that one post and it refreshes on the next feed request.

Write path: post → queue → fan-out worker reads followers via GSI → prepends postId to each follower's ZSET. Read path becomes a single ZSET read + hydration.

The celebrity problem

We traded read complexity for write amplification. One post by an account with millions of followers triggers millions of Redis writes — overwhelming the system and spiking write latency for the most popular users.

Great

Hybrid — fan-out-on-write for most, fan-out-on-read for celebrities

Set a follower threshold — say 100,000. Below it, precompute feeds as in the Good approach. Above it (the "celebrities"), don't precompute:

When a celebrity posts, write to the Posts table and skip the async feed fan-out.
When a user reads their feed: fetch the precomputed portion from Redis (posts from <100k accounts), then query recent posts from the celebrities they follow, merge chronologically, and return.

Result: an effective mix — write-time precomputation for the majority, read-time merging for the few celebrities. Fast reads for most users, manageable write amplification.

Trade-offs of the hybrid

The 100k threshold must be tuned — too low and write amplification returns; too high and read performance suffers for too many users.
Users following many celebrities get slower feeds → inconsistent UX; set per-segment SLAs.
Two systems to run (precomputed Redis + real-time celebrity queries) → more ops complexity and monitoring.

This is real

Instagram uses a similar hybrid in production — evidence the benefits outweigh the complexity.

Pattern: Managing Long-Running Tasks

Fan-out-on-write is the textbook long-running-task pattern: one post must update many follower feeds — slow work that must happen asynchronously off a queue, not inline with the request.

Instructor Insight — don't treat Redis as a magic black box

Proactively address durability. If Redis dies without persistence you lose all cached feeds. Show you've thought about failure modes: "We'd implement Redis with AOF persistence and Sentinel for failover to handle node failures with minimal data loss." Redis Cluster can shard across nodes; AOF or RDB snapshots enable recovery.

Community debate — Redis vs. DynamoDB for the feed store

A live disagreement in the comments the breakdown left open (there's no single right answer):

Redis-only: blazing in-memory reads, but volatile and memory-constrained. One commenter reported a Meta interviewer struggling to accept "no feed table, only a cache" — the lesson being to lead with durability (AOF/RDB, Sentinel) when you propose it.
DynamoDB / Cassandra-backed: persistent, effectively unlimited storage, slightly higher latency. Cassandra's clustering key keeps postIds sorted per user; write-skewed workload fits it well.
Both (common resolution): persist feeds durably and use Redis as a hot cache in front; enrich Redis from the store on miss. Feeds are also somewhat ephemeral — users rarely scroll past 2–3 days — so unbounded persistence may be unnecessary anyway.

Section takeaway

Fan-out-on-read fails on read amplification, repeated work, unpredictable latency at 150k+ req/s.
Fan-out-on-write (Redis ZSET + GSI + async queue) makes reads a single fast op — but creates the celebrity write-amplification problem.
The hybrid (write for <100k followers, read-merge for celebrities) is the accepted answer, and what Instagram actually does.

Deep Dive · Instant Media (≤ 8MB photos, ≤ 4GB video)

Two challenges with large media: upload efficiency and download/viewing latency. Take them one at a time.

Upload — chunking & pre-signed URLs

A single HTTP request is typically capped below 2GB, so a 4GB video needs at least two (in practice more, with metadata). The solution: AWS S3 multipart upload.

Multipart upload flow

POST /posts creates post metadata, returns a postId and a pre-signed URL (valid ~1 hour) that lets the client upload directly to S3, bypassing our servers.
Client uses the multipart upload API to send the file in chunks to that URL.
S3 reassembles the chunks and stores the final file.

Pattern: Handling Large Blobs

Pre-signed URLs for direct upload + chunked uploads for reliability + CDN for global download is the reusable large-blob recipe — it shows up in video platforms, file storage, and beyond.

Marking the upload complete

The post starts with media_status = pending. Once bytes land, set the S3 object_key and flip status to complete. Two ways:

Client-driven: client sends a PATCH when the upload finishes. Simpler, but trusts the client.
Server-driven: S3 event notification triggers a Lambda/background job that updates metadata. More complex, better consistency. Most production systems choose this.

Community footnote — you actually need multiple pre-signed URLs

The write-up simplifies to one pre-signed URL, but multipart upload really needs one pre-signed URL per part. Worth stating in a real interview to show depth. The commenters' consensus on who decides chunk count: prefer server-side — client sends the file size, the server decides segments and returns the list of pre-signed URLs. Reasons: avoids extra client↔server round trips, and you can't control when users update the app, so keep tunable/buggy logic server-side where you can patch it instantly. (A dissenting view favors client-side control for content-defined chunking; presigned URLs are generated in-memory via the S3 SDK, so they're cheap either way.)

Download — getting media to render fast

Bad

Serve directly from S3

Return the S3 URL and let the client download from S3. Fine for a small app; wrong for a global one.

Why it fails

Users far from the bucket region wait seconds (Singapore → us-east-1). Every request hits S3 (expensive, no caching). And everyone gets the same high-res file regardless of device or network — wasted bandwidth.

Good

Global CDN (CloudFront) in front of S3

Edge locations cache media near users; a request routes to the nearest edge, which fetches from S3 on miss and caches for later. Cache images ~24h (they rarely change), use built-in compression. Singapore now loads from a nearby edge, not the US East Coast.

Still not ideal

Same file for everyone — a slow mobile connection gets the same high-res image as a fast desktop. Limited control over caching means popular media can get evicted, forcing origin fetches.

Great

CDN + dynamic media optimization

On upload, a media-processing service (Cloudinary, Imgix) generates multiple variants: image resolutions + formats like WebP; for video, multiple quality levels with adaptive streaming (see the YouTube breakdown). The client sends device + network info, and the CDN serves the best-fit variant — mobile gets appropriately sized images, desktop gets high-res.

Add smart caching: cache popular media more aggressively, shorter TTLs for cold content, and proactively warm caches in regions where viral content is predicted to spike.

Cost of the win

Multiple variants per file → more storage and complexity; the processing pipeline adds upload latency and needs monitoring. Still, this is what large media platforms use — best balance of performance and manageability.

Section takeaway

Upload: multipart + pre-signed URL(s) → client uploads straight to S3; server-driven completion (S3 event → job) for consistency.
Download: CDN is essential; dynamic optimization (per-device variants, adaptive streaming, cache warming) is the great answer.

Deep Dive · Scaling to 500M DAU

We designed for scale throughout; here's the summary of what carries the load, plus the storage math.

Precomputed hybrid feeds — the cornerstone; precomputing for the majority slashes read-time load, with celebrity real-time merging as the deliberate write-amplification trade.
CDN — essential for global, low-latency media; dynamic optimization tunes for device/network.
Chunked uploads — reliability and parallelism for large videos.
DynamoDB + careful indexing — horizontal scalability for metadata; partition/sort keys keep queries efficient.

Storage math

Media: ~2MB avg × 100M posts/day = 200TB/day → ~750PB over 10 years. S3 handles it, but it's not cheap.
Metadata: ~1KB × 100M/day = 100GB/day of non-binary data.

Tiered storage — warmer = pricier

Move infrequently accessed media to cheaper cold storage like S3 Glacier; move cold metadata out of DynamoDB to S3. The ladder CDN → Cache/Memory → SSD → HDD → Tape lets you push cold data "down a level" to save money.

Throughput: horizontally scale the stateless microservices automatically on CPU/memory thresholds, each fronted by an implicit load balancer.

Instructor Insight — math with a purpose only

From the instructor's broader guidance: skip back-of-envelope math right after non-functional requirements if it doesn't change anything. "Most candidates do QPS/DAU/storage math, go 'wow, it's a lot,' and I learned nothing about the candidate — math without a purpose." Do calculations only when the result directly influences a design decision — usually in the high-level design or deep dives (e.g., "does this fit one instance or do we need to shard?").

What's Expected at Each Level

Mid-level

A working high-level design covering the core requirements: upload, follows, feed. A simple schema and how posts are stored/retrieved. May start with naive fan-out-on-read but, with hints, recognizes its limits and reaches fan-out-on-write. Grasps S3 + why a CDN matters, even without optimization detail.

Senior

Nails the deep dives — especially feed optimization and media delivery. Articulates the trade-offs between read / write / hybrid fan-out and why Instagram chooses hybrid for the celebrity problem. Strong tech justifications, detailed indexing strategy, optimizing for a read-heavy workload. May not cover every dive but can reason to a sound answer when asked.

Staff+

Identifies the true bottlenecks (feed generation + media delivery) fast and focuses there. Discusses system evolution — 1M → 500M users — knowing where simple solutions suffice and precisely when to evolve. Deep operational sense: anticipates failure modes, prioritizes user experience.

Concluding the interview

From the instructor: at the end, check your design against every functional and non-functional requirement. If it satisfies both lists, you can be confident. Naming out-of-scope features earlier shows product thinking and lets the interviewer reprioritize.

★

Active Recall

Answer before expanding. If you can't, revisit the linked section.

Which CAP property does Instagram prioritize, and why is that acceptable?

Availability over consistency. A post made in Germany not showing in a US feed for seconds-to-minutes is harmless, so eventual consistency (~2 min) is fine. The feed must stay available even if slightly stale.

Name the four core entities.

User, Post, Media, Follow. Post and Media are deliberately normalized/separated; one Post entity serves both photos and videos; Follow is uni-directional (follower → followee).

What are the three endpoints and why does the feed use a cursor?

POST /posts, POST /follows, GET /feed?cursor=&limit=. The cursor paginates efficiently and returns a next_cursor for the next page — better than offset at scale.

Why does fan-out-on-read break at Instagram scale? Give the three failure modes.

Read amplification (many DB reads per refresh), repeated work (same popular posts re-fetched millions of times), and unpredictable latency (varies with follow count/activity). At ~150k feed req/s you'd over-provision or miss 500ms.

What is fan-out-on-write, and which index makes it efficient?

Precompute feeds at post time: on a new post, find all followers and prepend the postId to each feed. Needs a GSI on Follows with followedId (partition) + followerId (sort) to look up followers fast.

How is a precomputed feed stored in Redis?

A Sorted Set (ZSET) keyed feed:{user_id}; members = postIds, scores = timestamp (or postId). Reading = one fast top-N ZSET read, then hydrate postIds into full post data.

What are the three hydration options, and which wins?

(1) store postIds only, fetch metadata from DynamoDB per request; (2) cache full metadata in Redis; (3) hybrid — ZSET of postIds + short-lived HASH metadata cache, batch-fetching misses via BatchGetItem. Option 3 gives the best performance/memory balance.

What is the celebrity problem?

Write amplification. One post by an account with millions of followers triggers millions of Redis feed writes, overwhelming the system and spiking write latency for popular users.

How does the hybrid feed approach solve it?

Threshold (~100k followers). Below: fan-out-on-write (precompute). Above (celebrities): don't precompute — on feed read, fetch the precomputed portion from Redis, then query recent celebrity posts and merge chronologically. Instagram does this in production.

What durability concern must you raise about Redis, and how do you address it?

If Redis dies without persistence, all cached feeds are lost. Use AOF or RDB persistence, Sentinel for automatic failover, and Redis Cluster to shard across nodes. Don't treat Redis as a magic black box.

How are large media files uploaded?

S3 multipart upload via pre-signed URL(s) — the client uploads chunks directly to S3, bypassing our servers. In reality you need one pre-signed URL per part. HTTP request size caps (<2GB) force chunking for 4GB videos.

Client-driven vs server-driven upload completion — which and why?

Server-driven is preferred in production: an S3 event notification triggers a Lambda/job that flips media_status to complete and sets the object_key. It's more complex but doesn't rely on trusting the client.

Walk the Bad → Good → Great progression for media download.

Bad: serve directly from S3 (high latency far from region, no caching, one-size file). Good: CDN in front of S3 (edge caching near users). Great: CDN + dynamic optimization — per-device variants, WebP/adaptive streaming, aggressive caching + cache warming for viral content.

Roughly how much media and metadata per day, and how do you control cost?

~2MB × 100M = 200TB/day media (~750PB/10yr); ~1KB × 100M = 100GB/day metadata. Push cold data down the ladder CDN → Memory → SSD → HDD → Tape — e.g., old media to Glacier, cold metadata to S3.

When should you do back-of-envelope math?

Only when the result changes a design decision (e.g., does data fit one instance or do we need to shard?) — usually during high-level design or deep dives. Math done just to say "wow, it's big" teaches the interviewer nothing.

Quick Revision Sheet

Requirements

FR: post · follow · chronological feed
Out: likes/comments, search, stories, live
NFR: avail > consistency (eventual ~2min); feed <500ms; media <500ms; 500M DAU
Scale: 500M DAU · 100M posts/day

Entities & API

User · Post · Media · Follow
POST /posts → postId + pre-signed URL
POST /follows (followerId from token)
GET /feed?cursor=&limit=

Indexes (DynamoDB)

Follows: PK followerId, SK followedId
Follows GSI: PK followedId, SK followerId (for fan-out)
Posts: PK userId, SK (createdAt, postId)
Real IG uses PostgreSQL for metadata

Feed — the answer

Read → Write → Hybrid
Redis ZSET feed:{user}, hydrate via HASH + BatchGetItem
Async: post → queue → fan-out worker
Celebrity (>100k): read-merge, don't precompute

Media

Upload: multipart + pre-signed URL(s), direct to S3
Completion: server-driven (S3 event → job)
Download: CDN → + dynamic optimization
Variants per device; adaptive streaming; cache warming

Scale & cost

200TB/day media · 100GB/day metadata
Glacier for cold media; S3 for cold metadata
Horizontal auto-scale on CPU/mem + LBs
Redis: AOF/RDB + Sentinel + Cluster

Non-functional requirement → what solves it

Requirement	Solution
Availability > consistency	Eventual consistency; DynamoDB; feed still serves if slightly stale
Feed < 500ms	Fan-out-on-write into Redis ZSETs; hybrid for celebrities; hydration cache
Media < 500ms	CDN edge caching + dynamic per-device optimization + cache warming
Scale 500M DAU	Precomputed feeds, CDN, chunked uploads, DynamoDB sharding, tiered storage, horizontal scaling

The one-line summary

Instagram is a scaling-reads + large-blobs problem. Precompute feeds with a hybrid fan-out (write for normal users, read-merge for celebrities), and deliver media through a CDN with dynamic optimization after pre-signed multipart uploads to S3.

Source & fidelity note

Built from the Hello Interview Instagram breakdown (written page as primary source) with the video's spoken commentary integrated as Instructor Insight callouts. Genuine open debates from the community — Redis vs. DynamoDB feed storage, and single vs. multiple pre-signed URLs — are preserved as unresolved rather than silently decided.