Video-on-Demand Architecture
The video-on-demand module — replays, highlights, fan uploads, a Netflix/YouTube inside your sports service. This chapter follows a video from the moment it lands in a bucket through a stateless processing workflow, out to edge caches worldwide, and into a data model that has to store a web of relationships between users, channels, and videos.
Assumes you know
After this you should be able to
- Explain the core idea in Video-on-Demand Architecture
- Apply Video-on-Demand Architecture in a representative design
The video-on-demand module — replays, highlights, fan uploads, a Netflix/YouTube inside your sports service. This chapter follows a video from the moment it lands in a bucket through a stateless processing workflow, out to edge caches worldwide, and into a data model that has to store a web of relationships between users, channels, and videos.
Two halves. First the data plane — how a video moves: upload → workflow → transcode → CDN → viewer, plus the storage estimates that justify every design choice. Then the data model — a long-running debate the chapter frames as "relationships live better in SQL than NoSQL," worked through both a relational schema and MongoDB's embedded vs. referenced models. Anything marked [+] is context I added; everything else is rewritten from the five teaching lessons.
What it must do, and the workflow that does it#
Four functional requirements, YouTube-style: (1) users upload videos via their channel (fan clips from the stadium/sports bar, plus official broadcaster channels); (2) watch a video from the VOD home page; (3) follow a channel and get notified of new uploads; (4) search the platform. Non-functional requirements — scalable, available, resilient — are a given for every service (a design that doesn't scale isn't a good design), stated explicitly only when special.
Live streams arrive at a standard resolution. User uploads come in every format, resolution, and dimension — so you need transcoding rules keyed to each video's metadata. A 144p clip needs no transcoding; a 4K video fans out into several lower resolutions; 15–30s shorts (Reels/YouTube Shorts) may get their own handling.
Two buckets, one workflow
Use two object-store buckets: a source bucket where uploads land temporarily, and a destination bucket where transcoded videos end up. (One bucket works too, but two is cleaner.) A video landing in the source bucket triggers a workflow.
A DAG — a Directed Acyclic Graph of tasks, with directed edges and no cycles — used to automate processes, typically via message queues. It gives you an entry point plus failure handling, retries, rollbacks, parallel execution, and observability. Apache Airflow is a common tool for defining them.
Three stateless functions fired by the DAG#
When a video hits the source bucket, the workflow triggers serverless functions:
- Metadata extraction — pulls resolution, frame width/height, length, bitrate, audio sample rate,
etc., and writes them to a relational
VideoMetadatatable (VideoId, Name, Title, Description, UploadDate, ChannelId, SportsCategory, VideoLikes, Views, VideoSize, Resolution, DiskLocation, …). - Thumbnail generator (runs in parallel) — creates a thumbnail, stores the file in a
VideoThumbnailBucket, and adds its path to a newThumbnailDiskLocationcolumn. One thumbnail per video keeps it simple; multiple would need a separateThumbnailtable (one-to-many). - Transcoding — pulls the video from the source bucket, creates the transcoded versions, and stores them in the destination bucket.
Extracting metadata, generating a thumbnail, or transcoding needs nothing beyond the video itself — not the uploader's identity or history. Stateless events mean instances don't idle holding state; they spin up only when a video needs processing. That saves money. (Later workflow steps: populate the CDN, push to viewers' home pages, and send notifications.)
Delivery: replicate to the edge, cache in layers#
Videos from the destination bucket are intelligently replicated to edge locations — a video popular in a region goes to that region's edges. Replicating every popular video to every edge globally would be overkill.
- A viewer clicks a video; the request routes to the nearest edge location.
- Edge checks its cache. Hit → return to viewer.
- Miss → fetch from origin, cache it if it's popular enough per business rules, and return to the viewer.
Add a regional cloud cache between the edge and the origin. It holds videos less popular than edge content; as edge videos cool off over time, they demote to this secondary cache. It significantly cuts the requests reaching the origin data center.
The numbers: 900 PB streamed, 212 TB uploaded — daily#
The exercise isn't about exact figures — it's about how you reason to them. Assume 300M subscribers, 100M daily active, each watching ~3 hours/day.
Daily bandwidth consumed
At HD 1080p ≈ 3 GB/hour → 9 GB per viewer/day × 100M viewers = 900 Petabytes/day streamed.
Daily uploads & storage
Assume an upload:consumer ratio of 1:1000 → 100K uploads/day. Split them 10% 4K / 80% 1080p / 10% shorts, and add every transcoded version:
| Upload tier | Count/day | Storage incl. transcodes |
|---|---|---|
| 4K, 1 hr (10%) | 10K | 7 GB each → 70 TB; +1080p 30 TB +720p 10 TB +480p 7 TB = 117 TB |
| 1080p, 15 min (80%) | 80K | 0.75 GB each → 60 TB; +720p 20 TB +480p 14 TB = 94 TB |
| 720p, 15 sec (10%) | 10K | ~0.042 TB; +480p ~0.013 TB = ~0.055 TB |
| Total (one copy) | 100K | ≈ 212 TB/day |
| ×3 regions (redundancy) | ≈ 636 TB/day |
Thumbnails and metadata are omitted as negligible. Resolution/bitrate references come from Netflix (4K = 7 GB/hr, 1080p = 3 GB/hr, 720p = 1 GB/hr, 480p = 0.7 GB/hr). Without historical data, any such estimate is rough — the method is the lesson.
Modeling the user side (relationally)#
Why relational, not a document DB like MongoDB? Because the data is deeply relational: a user owns multiple channels, subscribes to multiple channels, a video has many comments and thumbnails. Relational databases specialize in relationships. The schema:
A user relates to channels two ways: ownership (User → UserChannel, one-to-many) and
following (User ↔ Channel, many-to-many via the ChannelSubscription bridge). And
Channel has a one-to-many with VideoMetadata (ChannelId as foreign key), since a
channel holds many videos.
Ideally a graph model fits many-to-many best (covered in a later chapter), but here it's done relationally. Which raises the question the next lesson tackles: can a document DB store these relationships too?
Relationships in a document DB: embedded vs. referenced#
In a document DB (MongoDB), a document = a relational row and a collection = a table. It stores relationships two ways. Take the many-to-many User ↔ Channel:
| Model | How it works | Trade-off |
|---|---|---|
| Embedded | Each Channel document embeds all its subscribers; each User document embeds all channels they subscribe to. | Highly denormalized — huge duplicate data (a user with 100 channels × 300M users), and writes must update many places, risking inconsistency. But reads fetch everything in one query (no join). |
| Referenced | Separate User, Channel, and ChannelSubscription collections with references (foreign keys) — mirrors the relational design. | Normalized. Fetching across collections needs extra queries, just like relational joins — so performance ends up equivalent to the relational approach. |
Embedding is wrong for the many-to-many subscription case, but great for a one-to-many like User → UserChannel (channels a user owns): embed that list in the User document and you avoid a join entirely. In NoSQL, you model data around the queries you expect — which usually ends in denormalization, precisely to cut read latency. That's the whole reason you'd reach for NoSQL.
[+] The relational world does the reverse: normalize first, then optimize reads with minimal joins and query caching. But nothing forbids denormalizing in SQL too — if it cuts latency while preserving integrity, do it.
So which database wins?#
NoSQL databases support relationships but aren't built for them — they're built to access data with minimum latency, which works best when data is denormalized. A relational database specializes in relationships, has decades of maturity behind it, and gives strong consistency — imperative for financial transactions. When dealing with relationships, stick with relational. (The same relational store will back the subscription-payment module in a later chapter.)
With the upload-and-delivery workflow designed and the data modeled, the next chapter recommends videos to viewers on the VOD home page via a video feed service.
Recap#
A VOD system is two pipelines that never touch. The bytes pipeline — the actual video — wants to be dumb, stateless, and cached as close to the viewer as possible; you push it out to the edge and let serverless functions chew through it on the way. The relationships pipeline — who owns what, who follows whom — wants structure and consistency, so it lives in a relational database. Confusing the two (video files in a database, or relationships in a denormalized document store) is where these designs go wrong. Keep the heavy dumb data at the edge and the connected smart data in SQL.
Further reading#
All three external references resolve live. The author's own course cross-links are omitted.