Designing a File Storage Service Like Dropbox
Build a system that lets users upload, download, share, and automatically sync files across devices — cheaply, reliably, and even when a single file is 50 GB. The interesting engineering hides in three places: how large files move, how you keep two folders consistent, and how you avoid moving bytes through your own servers.
Evan (ex-Meta staff engineer) has asked this question roughly 50 times. It's officially "on the easier side" and shows up most for mid-level (E4/L4), but it's asked all the way up to staff. It's really a test of whether you can reason through file transfer at scale — bypassing app servers, resuming interrupted uploads, and keeping distributed copies in sync.
1Understanding the Problem
A cloud-based file storage service that lets users store and share files, and access them securely from any device. You keep a copy of each file locally (in a synced folder) and remotely (in the cloud); the system's job is to keep those in agreement.
Functional requirements — the features
Work through these one at a time; that ordering becomes your design plan. The core features:
- Users can upload a file from any device.
- Users can download a file from any device.
- Users can share a file with others and see files shared with them.
- Files automatically sync across devices.
Out of scope: editing files, previewing without download, per-user storage limits, versioning, virus scanning — and, importantly, rolling your own blob storage (that's a separate "Design S3" question).
In the video, Evan defines only three functional requirements — upload, download, sync — and treats sharing as a side note. The written article promotes sharing to a full fourth requirement with its own design ladder. Both are defensible; just be explicit with your interviewer about which features you're committing to build.
2Non-Functional Requirements & the CAP Trade-off
Non-functional requirements are the qualities of the system. Most candidates breeze through these with generic buzzwords — but for senior and staff levels these are exactly what should lead your deep dives later. Phrase each one in the context of this system, and quantify where you can.
- Availability > consistency. Prioritize being able to serve files over everyone seeing the very latest version instantly.
- Support large files — up to 50 GB (exactly what real Dropbox supports), with resumable uploads.
- Secure & reliable — recoverable if files are lost or corrupted; high data integrity / sync accuracy.
- Low latency upload, download, and sync — "as fast as possible" (hard to put a single number on, since files vary).
You should prioritize consistency over availability only when every read must reflect the most recent write or the system breaks. A stock-trading app is the classic case: if a user buys AAPL in Germany, a buyer in the US must see that trade replicated before proceeding. File storage has no such rule — if someone uploads a file in Germany and a user in the US can't see it for a few seconds, nothing breaks. So we accept eventual consistency and keep the system highly available.
Evan deliberately does not do back-of-the-envelope math up front. Storage sits on near-infinitely-scalable blob storage (S3), so a headline "wow, that's a lot of data" number doesn't inform any decision. His rule: do a calculation only at the moment its result will directly change your design (you'll see exactly that when we compute upload time for a 50 GB file). Tell your interviewer you're taking this approach — they rarely object.
The video adds sync accuracy / data integrity as a named requirement: eventual consistency is fine and a couple seconds of lag is acceptable, but once things settle, what's in the local folder must equal what's in remote must equal what's in every other folder. This requirement is what later justifies the reconciliation job.
3Core Entities & the API
Core entities
Start broad — you don't need every column yet, just the objects that get persisted and exchanged. For Dropbox they're almost trivial:
- File — the raw bytes users upload/download/share.
- FileMetadata — name, size, MIME type, uploader, and (later) the S3 link, status, and chunk list.
- User — the account. So minor here that the video suggests it can be more distraction than help.
The API — one endpoint per feature (first draft)
Use the core entities as inputs/outputs so you can move fast. Upload and download start simple:
POST /files
Request: { File, FileMetadata }
GET /files/{fileId} -> File & FileMetadataSharing, and a way for clients to discover what changed since they last synced:
POST /files/{fileId}/share
Request: { User[] } // users to share with
GET /files/changes?since={timestamp} -> ChangeEvent[]Each ChangeEvent carries the fileId, the change type (created / updated / deleted), and updated metadata. The since timestamp lets a client fetch only what changed since its last sync.
Evan flags up front that this first-draft upload API will not survive. You won't actually POST a 50 GB file body through your own server — you'll switch to presigned URLs and chunk uploads. Say this out loud: "I'll outline simple APIs now and revise them as we weigh trade-offs." Most mid-level candidates don't know the final shape yet, and that's fine — the reasoning back-and-forth is the point.
Pass the user via an auth header (session token or JWT), never in the request body. A body value is trivially forged, letting anyone upload or download on another user's behalf.
4High-Level Design
The goal of the high-level design is simply to satisfy the functional requirements — so walk your APIs one by one. For each feature the source builds a Bad → Good → Great ladder; the reasoning between rungs is what an interviewer is grading.
Two storage questions drive everything: where do the raw bytes live, and where does the metadata live? Metadata is loosely structured with a dominant "fetch files by user" query, so a NoSQL store like DynamoDB fits — though PostgreSQL works just as well. A starter metadata document:
{
"id": "123",
"name": "file.txt",
"size": 1000,
"mimeType": "text/plain",
"uploadedBy": "user1"
}Evan skips arguing SQL vs NoSQL: "these technologies have converged — well-configured, they can largely do the same things." With SQL you'd normalize chunks into a separate table with a foreign key; with DynamoDB you'd embed the chunk list on the item. He picks DynamoDB and moves on. Don't burn interview time here.
4.1 Upload — from local disk to blob storage
Store files on the app server's disk
The File Service accepts the file + metadata and writes the bytes to its own local filesystem.
Why it failsStorage fills up as files grow, forcing constant scaling; and if the server dies you lose every file. Not scalable, not reliable.
Server forwards the file to S3 / GCS
The backend receives the upload and pushes it to blob storage, saving metadata in the DB. Near-infinite scale and durability come for free.
Why it's not enoughYou upload the file twice — once to your server, once to S3 — wasting bandwidth and CPU. You also must handle the partial-failure case (file saved but metadata not, or vice-versa) with a transactional "only commit metadata if the upload succeeded" approach.
Client uploads straight to S3; backend only signs
The POST /files becomes a three-step dance:
- Client requests a presigned URL (backend saves metadata with
status: "uploading"). - Client
PUTs the bytes directly to that URL — the file never touches your servers. - S3 fires an S3 Notification to your backend, which flips
statusto"uploaded".
POST /files/presigned-url -> PresignedUrl
Request: { FileMetadata }Why it winsFaster and cheaper — bytes flow client→S3 with no redundant hop. This is the classic "Handling Large Blobs" pattern: bypass app servers for data transfer, use signed URLs for security.
A URL your backend generates (using its AWS credentials and the S3 SDK) that grants a client permission to upload/download a specific object for a limited time. Generating it is a purely local, cryptographic operation — no call to S3 is made. The signature can pin the allowed MIME type, size, and expiry (e.g. 10–30 min), so the holder can only do exactly what you authorized.
4.2 Download — get bytes back to the user
S3 → backend → client
The file is downloaded once from S3 to your server, then again from your server to the client — slow and expensive (the mirror image of the redundant-upload problem).
Presigned download URL
GET /files/{fileId}/presigned-url returns a time-limited link; the client pulls bytes straight from S3.
Remaining weaknessS3 lives in one region, so users far away get slow downloads.
Serve from the nearest edge
A CDN (e.g. CloudFront) caches files close to users. The File Service hands out a CDN signed URL instead of a raw S3 URL; the CDN fetches from S3 on a cache miss and serves from the edge afterward.
The costCDNs are expensive. Use cache-control headers and invalidation so only frequently-accessed files are cached.
This is where the video pushes back on the article. A CDN only pays off when the same file is read many times, or read far from where it's stored. But Dropbox users mostly download their own files, and they're usually near the region that holds them. So the CDN mainly helps for heavily shared or frequently-traveled files. Evan would likely leave the CDN out and note the trade-off — and the article's own comment section agrees, pointing out that a CDN is a cache that only helps read-heavy, publicly-shared files, which are a small fraction of Dropbox traffic.
4.3 Sharing — the fast lookup problem
Sharing works like Google Drive: enter an email, grant access. The design question is how to quickly list the files shared with a user when they open the app.
Add a sharelist array to each file's metadata
Simple to write. But to show "files shared with me," you'd have to scan every file's sharelist — there's no index for the inverse lookup.
Cache a user → files mapping
Keep the sharelist and a cached inverse (user1: ["fileId1","fileId2"]) so "shared with me" is a fast key lookup.
The catchTwo structures to keep in sync — best done by storing both in the same DB and updating them in one transaction.
A dedicated SharedFiles table
Map userId (partition key) → fileId (sort key). Listing shared files is a single indexed query, and there's no sharelist to keep in sync.
| userId (PK) | fileId (SK) |
|-------------|-------------|
| user1 | fileId1 |
| user1 | fileId2 |
| user2 | fileId3 |Trade-offSlightly slower than a raw key-value lookup (it's an index query), but eliminates the sync burden — usually worth it.
4.4 Sync — keeping local and remote in agreement
A copy of each file lives locally and in the cloud (the source of truth). Sync runs in two directions:
Most system-design diagrams shrink the client to a tiny box. For Dropbox (like Netflix/Spotify/YouTube) there's real logic on the client, so draw it big. It contains: the synced local folder, a downloadable client app, and a local DB that stores metadata (IDs, fingerprints) for files already downloaded — so the client can tell whether a "changed" remote file is one it already has locally.
Local → Remote
A client-side sync agent watches the folder using OS file-system events — FileSystemWatcher on Windows, FSEvents on macOS. On a change it queues the file, calls the upload API with updated metadata, and updates the remote's S3 link + updatedAt. Conflicts resolve by last-write-wins.
Remote → Local
Each client must learn when the server changed so it can pull. The client also cross-references the local DB to avoid re-downloading files it already holds.
This is the sharpest conflict between the two sources — worth knowing both:
Article: recommends a hybrid — one WebSocket (or SSE) per device for near-instant push, plus periodic polling (GET /files/changes?since=…) as a safety net for dropped connections. "Best of both worlds."
Video: Evan argues adaptive polling alone is appropriate and that WebSockets/long-polling are overkill — a persistent always-on connection per user adds a WebSocket manager and real overhead, and the product doesn't need millisecond freshness. A manual "refresh" button covers the impatient case.
Neither is wrong; it's a genuine product/latency trade-off. Pick a side and state why.
5Tying It All Together
The pieces assembled into one system that satisfies every functional requirement:
- LB & API Gateway — routing, SSL termination, rate limiting, auth. (AWS API Gateway caps request bodies at ~10 MB — remember this number.)
- File Service — the control plane: reads/writes metadata and signs presigned URLs. It never carries file bytes.
- Metadata DB — file metadata + the shares table + (later) per-chunk status.
- S3 — the actual bytes; clients read/write it directly via signed URLs.
- CDN — optional edge cache for downloads.
6Deep Dive · Supporting Large Files (up to 50 GB)
This is the meat of the problem — where interviewers spend the most time. Two UX goals frame everything: a progress indicator and resumable uploads (don't make someone re-send 49 GB after a dropped connection).
Why a single POST can't work
- Timeouts — a huge single request blows past client/server timeout windows.
- Payload limits — browsers, servers, and API gateways cap body size. AWS API Gateway's hard limit is 10 MB and can't be raised.
- Network interruptions — one drop means starting over.
- No progress — the user stares at a spinner blind to what's happening.
A 50 GB file over a 100 Mbps connection:
50 GB × 8 bits/byte ÷ 100 Mbps = 4000 s → 4000 ÷ 60 ÷ 60 ≈ 1.11 hours
An hour-plus with no feedback and no recovery is unacceptable — this is the calculation worth doing in the interview, because its result forces the chunking design.
Break the file into 5–10 MB chunks on the client and upload each directly to S3 (serially or in parallel). Chunking gives you three things at once: a natural progress bar, resumability, and better bandwidth use. A very common mistake is chunking on the server — that defeats the purpose, since you still had to ship the whole file to the server first.
Tracking chunk status for resumability
Store per-chunk state on the FileMetadata item so a resumed upload only sends what's missing:
{
"id": "123", "name": "file.txt", "size": 1000,
"mimeType": "text/plain", "uploadedBy": "user1",
"status": "uploading",
"chunks": [
{ "id": "chunk1", "status": "uploaded" },
{ "id": "chunk2", "status": "uploading" },
{ "id": "chunk3", "status": "not-uploaded" }
]
}Identifying files & chunks: fingerprinting
A hash (e.g. SHA-256) computed over a file's — or chunk's — content. It uniquely identifies content regardless of filename or source, enabling deduplication and resumability. To resume, compare local chunk fingerprints against the fingerprints already marked "uploaded" server-side, and send only the gaps.
Two different users uploading the identical file produce the same fingerprint. So your fileId should be a unique UUID (the record), while the fingerprint is a separate field used only for dedup/resume checks. Don't naively key files by name — different users (or the same user) reuse names constantly.
Keeping chunks in sync with reality
Client reports each success
Client uploads a chunk to S3, gets a success, then PATCHes the backend to mark that chunk uploaded.
The riskYou're trusting the client. A malicious user could mark chunks uploaded without uploading them — corrupting only their own file, but leaving your metadata inconsistent with S3, which is a debugging nightmare.
Server confirms with S3 via ETags
Each chunk gets an ETag on upload. The client includes ETags in its PATCH; the backend verifies them against S3's ListParts API before committing "uploaded." You accept client updates for instant progress feedback, but confirm server-side before finalizing.
You could let S3 Notifications (a form of change-data-capture) tell your backend when a chunk lands, avoiding client trust entirely. The gotcha: with S3's Multipart Upload API, notifications fire only when the whole object is finalized — not per part. So for per-chunk progress you must poll ListParts. Evan lands on trust-but-verify as his interview default, but calls both valid.
1) Client chunks the file (5–10 MB) and fingerprints each chunk + the whole file. 2) Client asks if a file with that fingerprint already exists — if it's "uploading", resume from existing chunk statuses. 3) Otherwise backend calls S3 CreateMultipartUpload, saves metadata as "uploading", returns an uploadId + a presigned URL per part. 4) Client uploads each part, PATCHes status+ETag, backend verifies via ListParts. 5) When all chunks are up, backend calls CompleteMultipartUpload; only after S3 confirms assembly does it mark the file "uploaded".
Once CompleteMultipartUpload runs, S3 assembles the parts into one object — downloads then behave like any normal file via a single (CDN or S3) signed URL. For very large files, HTTP Range requests let a client fetch byte ranges in parallel or resume an interrupted download, with no knowledge of the original chunk boundaries.
S3's Multipart Upload feature (and its JS SDK) does exactly this chunking/uploading/assembly for you — mentioning it signals hands-on experience. But you can't get away with only "I'd use multipart upload." Be ready to explain how you'd implement it yourself.
7Deep Dive · Making Upload / Download / Sync Fast
Much of the speed work is already done. To recap the levers, plus two more:
Parallel chunks max out fixed bandwidth
Bandwidth is fixed — the pipe is only so big — but sending multiple chunks in parallel with adaptive chunk sizes uses all the bandwidth available. The same chunking also powers efficient sync: when a file changes, transfer only the changed chunks.
With fixed 5 MB boundaries, inserting one byte near the start shifts every later boundary, so every subsequent chunk gets a new fingerprint — delta sync becomes useless. The fix is Content-Defined Chunking (CDC): boundaries are set by content using a rolling hash (Rabin fingerprinting), so a small edit only touches the chunks around it. This is how real Dropbox achieves efficient delta sync.
Compression — transfer fewer bytes
Compress on the client before upload; decompress after download (keeps the backend out of the data path). But compress selectively:
| File type | Compresses? | Worth it? |
|---|---|---|
| Text / DOCX | Very well (5 GB → ~1 GB) | Usually yes |
| Media (JPEG, PNG, MP4) | Already compressed — a few % at best | No — CPU cost outweighs it |
Put logic on the client to decide based on file type, size, and network conditions, and record the algorithm in metadata so you can decompress. Common choices: Gzip (universal), Brotli (better text ratios), Zstandard (excellent speed/ratio balance — a strong client-side pick).
Always compress before encrypting. Encryption randomizes the bytes, which destroys compressibility — so encrypt-then-compress yields almost no savings.
8Deep Dive · File Security
- Encryption in transit — HTTPS everywhere (table stakes).
- Encryption at rest — enable S3 server-side encryption; S3 encrypts each file with a key stored separately from the data.
- Access control — the shares table/cache is your ACL; only issue download links to authorized users.
Someone could post a download link to a public forum. This is why signed URLs expire quickly (e.g. 5 min). But note: a signed URL is a bearer token — anyone holding a valid, unexpired one can download. Expiry limits exposure but doesn't fully prevent sharing. For higher security, add IP binding or require auth cookies alongside the signed URL.
How signed URLs work end-to-end: the server generates a signature over the path + expiry (+ optional IP) using its private key; it's handed to the authorized user; the CDN/S3 validates the signature with the matching public key and checks expiry before serving.
9Deep Dive · Sync Speed & Consistency
Two properties matter: sync should be fast (remote changes appear locally quickly, within reason) and consistent (local and remote eventually match exactly).
Making sync fast
Adaptive polling. The client polls for changes, and the interval adapts — poll more often when the app is open or the folder is active, less often otherwise. A manual refresh button forces an immediate poll when needed.
Delta sync. On a change to an existing file, download only the changed chunks (not the whole 50 GB), update the local DB, and let the client app stitch the file back together. A brand-new file has no choice but a full download.
Making sync consistent: two options for "polling"
Query metadata directly
"For folder 123, give me any file whose chunk changed or was added since my last sync." Simple, effective, and Evan's default choice.
What Dropbox actually does
Push every change event onto an event bus (e.g. Kafka). Each synced folder stores a sync cursor marking the last event it consumed. First sync replays all events to build local state; later syncs read only events after the cursor. Add partitioning by user/folder and periodic snapshots so you don't replay everything forever.
An event bus with a cursor shines when you need an audit trail, versioning, and rollback — you can roll events forward or back to reconstruct any prior state. But those weren't in our functional requirements. If you overwrite a chunk in the plain-DB approach, the old chunk is gone and you can't roll back — which is fine given our scope. Evan deletes the event bus from his design and opts for simplicity, while flagging that if versioning were required, this is the pattern.
The safety net: reconciliation
Despite best efforts, bugs and dropped connections can leave local and remote inconsistent. Periodically (daily or weekly) the client app fetches all remote info for the folder, compares fingerprints against what's in the local DB and folder, and fixes any mismatch. This is the backstop that ultimately guarantees the "high data integrity" requirement.
Recall the "wrong" upload API from step 3. It's now fixed: POST /files with metadata → returns a presigned URL; client uploads chunks to that URL; client PATCHes chunk status as it goes. No file body ever passes through your servers.
10What's Expected at Each Level
| Level | Breadth / Depth | The bar for Dropbox |
|---|---|---|
| Mid (E4/L4) | ~80% breadth | Clear API + data model; a working high-level design for upload/download/share. Not expected to know presigned URLs or chunking up front — but should reason to them when nudged ("you're uploading twice — how do you avoid that?"). Interviewer may drive the later stages. |
| Senior (E5) | ~60/40 | Breeze through the HLD, then spend real time on large-file uploads. Proactively weigh options and land somewhere reasonable. Often can speak first-hand about APIs like multipart upload. Drive 1–2 deep dives. |
| Staff+ | ~40/60 depth | Deep dives on the hard scenarios, driven independently — interviewer intervenes only to focus, not steer. May steer toward a topic they find interesting. Fluent on trade-offs, treating the interviewer as a peer. Go deep in 2–3 places. |
✦Active Recall
Answer each before expanding. These test reconstruction of the reasoning, not trivia.
Q1Why does Dropbox prioritize availability over consistency, and when would that choice be wrong?
Because no read must reflect the latest write for the system to function — a few seconds of staleness when a file uploaded in one region isn't yet visible in another is acceptable. It would be wrong for a system where every read must see the most recent write, e.g. stock trading, where a share bought in one region must be replicated before a buyer elsewhere can act.
Q2Walk through the three-step direct-to-S3 upload. What problem does it solve versus routing bytes through your backend?
(1) Client requests a presigned URL; backend saves metadata as "uploading." (2) Client PUTs bytes directly to that URL. (3) S3 notifies the backend, which marks status "uploaded." It eliminates the redundant double-upload (client→server→S3), saving bandwidth and CPU, and it removes your servers from the data path entirely.
Q3Why must chunking happen on the client, and what breaks if you do it on the server?
Chunking exists to avoid shipping the whole file at once and to enable resumability/progress. If you chunk on the server, you've already uploaded the entire file to the server to do so — defeating the purpose, and you still hit payload limits and timeouts on that initial transfer.
Q4What is a fingerprint, and why store it separately from fileId?
A content hash (e.g. SHA-256) of a file or chunk, used for dedup and resumability. It identifies content, so two users uploading the same file share a fingerprint. fileId must stay a unique per-record UUID; the fingerprint is a separate field used only for dedup/resume comparisons.
Q5Explain "trust but verify" for chunk status. What's the alternative and its limitation?
The client reports each chunk uploaded (for instant progress), but the backend verifies ETags against S3's ListParts before committing — so a lying client can't corrupt metadata. The alternative is S3 notifications (CDC), which need no client trust, but S3 multipart uploads only fire a notification when the whole object finalizes, not per part, so per-chunk progress still needs ListParts.
Q6Fixed 5 MB chunks make delta sync nearly useless. Why, and what's the fix?
Inserting one byte near the start shifts every later boundary, so every subsequent chunk fingerprint changes and you'd re-transfer almost the whole file. Content-Defined Chunking (CDC) sets boundaries by content via a rolling hash (Rabin), so a small edit only affects nearby chunks.
Q7The article recommends WebSockets + polling; the video prefers polling alone. Reconstruct both arguments.
Article (hybrid): a persistent WebSocket/SSE gives near-instant push; periodic polling is a safety net for dropped connections and missed messages, guaranteeing eventual consistency. Video (polling only): a persistent per-user connection adds a WebSocket manager and overhead the product doesn't need — seconds-level freshness is fine, and adaptive polling + a manual refresh button cover it. It's a latency/complexity trade-off, not a right/wrong.
Q8When is an event-bus-with-cursor worth the complexity over polling the DB, and why did the video drop it?
It's worth it when you need an audit trail, versioning, and rollback — you can replay/roll events to reconstruct any state. The video drops it because those weren't in the functional requirements; overwriting a chunk in the simple DB approach loses history, which is acceptable given the scope. It's what real Dropbox does, but it's overkill for the stated requirements.
Q9Why compress before encrypting, and when should you skip compression entirely?
Encryption randomizes bytes, which are then nearly incompressible — so compress first. Skip compression for already-compressed media (JPEG/PNG/MP4): the gain is a few percent and doesn't justify the compress/decompress CPU time. Text/DOCX compress well and are usually worth it.
✦Quick Revision
Core Ideas
- Availability > consistency — no read must see the latest write.
- Bytes bypass your servers — client ↔ S3 via presigned URLs.
- Chunk on the client → progress, resume, parallel bandwidth.
- Fingerprints identify content for dedup/resume.
- Reconciliation is the ultimate integrity backstop.
Numbers & Rules
- Max file: 50 GB. Chunk size: 5–10 MB.
- API Gateway body cap: 10 MB (hard).
- 50 GB @ 100 Mbps ≈ 1.11 hours.
- Hash: SHA-256. Signed URL expiry: ~5 min.
- Compress before encrypt. Last-write-wins on conflict.
Bad → Great ladders
- Upload: server disk → via backend → direct-to-S3.
- Download: via server → direct S3 → CDN.
- Share: sharelist → inverse cache → normalized table.
- Chunk status: client PATCH → trust-but-verify.
Common Mistakes
- Chunking on the server.
- Keying files by name instead of fingerprint.
- Fixed chunk boundaries → broken delta sync (use CDC).
- Blindly adding a CDN without justifying reads.
- Generic, unquantified non-functional requirements.
Must Remember · One-line system
The File Service is a control plane that only signs URLs and records metadata; raw bytes flow client ↔ S3 directly. Large files are chunked and fingerprinted on the client, uploaded via multipart with trust-but-verify status, and delta-synced back down. Consistency comes from adaptive polling plus periodic reconciliation.
Study notes compiled from the Hello Interview write-up "Design a File Storage Service Like Dropbox" by Evan King, with instructor commentary integrated from the accompanying video walkthrough. Where the two sources disagree (sync notifications, CDN necessity, scope of "sharing"), both positions are preserved rather than reconciled.