Handbooks / System Design Case Studies / Chapter 10

Design a Video Streaming Platform like YouTube

12 pages · ~17 min✓ Reviewed

Builds on Design Cloud File Storage like Dropbox. Next up: Design a Payment System.

System Design Case StudiesPage 1

Problem Statement, Scope and Requirements

1. Scope and Functional Needs (Video Streaming Platform)

	Features
In scope	upload, transcode, stream, search metadata, view counts, recommendations
Out of scope	live streaming, DRM licensing, ads, comments

Upload a video
Watch on any device
Search videos by metadata
Like/comment-free watch history
Resume playback from the last position

2. Non-Functional Targets

Playback start under 2 s at p95
Rebuffer ratio under 0.5%
Playback availability 99.99%
Uploads durable at 11 nines
Reads dominate: read:write about 5000:1

3. Back-of-Envelope Estimates

Daily numbers, each step feeds the next

1500M DAU5 videos each
22.5B plays/day/ 86,400 s
329K play QPSpeak about 90K
4250 PB/day egress2.5B x 100 MB

System Design Case StudiesPage 2

High-Level Architecture

Running example: Video Streaming Platform at 500M DAU, ~90K peak play QPS, read:write about 5000:1.

1. Two Planes, Two Paths

	Write path	Read path
Steps	upload, transcode, publish	metadata, manifest, CDN segments
Load	~500K uploads/day	~2.5B plays/day, 90K peak QPS

System Design Case StudiesPage 3

Upload Pipeline

1. Responsibilities and Design (Video Streaming Platform)

Accept large files reliably: 500 MB average raw, 500K uploads/day
Validate the file, then store the raw original durably (11 nines)
Only then trigger processing (transcode, thumbnails, indexing)

Client uploads straight to object storage via a pre-signed URL
Chunked multipart: 5-10 MB parts, sent in parallel
API servers never carry bytes: they only issue URLs and track state
500 MB at 8 MB parts is about 63 parts; 250 TB/day is about 23 Gbps ingest

System Design Case StudiesPage 4

Transcoding Pipeline and Renditions

1. Responsibilities and Design

Video Streaming Platform: one raw upload goes in, a ladder of playable renditions comes out.

Convert one raw file into a ladder of renditions
Plus audio tracks and thumbnails
Ladder example: 240p at 0.4 Mbps up to 4K at 16 Mbps (about 6 rungs)

Rung	Bitrate (est.)

System Design Case StudiesPage 5

Adaptive Bitrate Streaming: HLS and DASH

1. What ABR Is and What the Manifest Lists

ABR = video cut into 2-6 s segments, one set per bitrate
The player picks the next segment's rendition only, never the whole video
Inputs: measured throughput and buffer level (seconds of video queued)
HLS: .m3u8 playlists, master playlist → media playlist per rendition
DASH: one XML .mpd file; both list renditions and segment URLs

text

#EXTM3U
#EXT-X-STREAM-INF:BANDWIDTH=3000000,RESOLUTION=1280x720
720p/index.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=800000,RESOLUTION=854x480
480p/index.m3u8

HLS master playlist: one entry per rung, each points to a media playlist

System Design Case StudiesPage 6

CDN Delivery and Caching

(Video Streaming Platform) 250 PB/day egress is about 23 Tbps average, so the CDN carries the read path.

1. Responsibilities and Tiers

Serve segments from close to the viewer
Target 95%+ edge hit ratio
Origin sees under 5% of 23 Tbps, about 1.15 Tbps
Netflix places Open Connect appliances inside ISPs

Tiered caching

1Edge POPclosest to viewer
2Regional mid-tierbigger, shared
3Origin shieldone per region
4Object storesource of truth

System Design Case StudiesPage 7

Metadata Service and Search

1. Responsibilities and Design

Store title, description, tags, owner, visibility and rendition pointers
Serve the watch page and search
Read-heavy: about 90K peak reads/s (5000:1 read:write)
With a 99% cache hit, the DB sees only about 900 reads/s at peak

Store: sharded SQL/NoSQL by video_id (MySQL + Vitess, or Cassandra)
Cache: Redis in front, 99% hit target
Search: Elasticsearch index fed by change events (CDC)

System Design Case StudiesPage 8

View Counts and Watch History

1. Responsibilities and Design

Count views accurately enough for display and creator payouts
Track the resume position per user and video
Load: 29K view starts/s avg, 90K peak
Heartbeats (~1 per 30 s) give ~5x events, so ~150K events/s avg

Players send view and heartbeat events to Kafka
Flink aggregates per video in 1-min windows
Counter store is updated in batches, never per event

System Design Case StudiesPage 9

Recommendations (High Level)

1. Job and Funnel

Home feed and up-next list for the Video Streaming Platform, ranked per user within a 150 ms p99 budget.

Ranks the personalized home feed and the up-next list
Drives most watch time, so it must be fast and fresh
Candidate generation: collaborative filtering / two-tower embeddings
Ranking: model scoring on watch time and CTR features

System Design Case StudiesPage 10

Scaling, Failure Handling, Trade-offs and Interview Questions

1. Scaling Levers (Video Streaming Platform)

Layer	Lever	Number
Metadata DB	Shard by `video_id`	~90K peak reads/s
Transcode	Autoscale on queue depth	time-to-ready under 2x duration
CDN	Absorbs 95%+ of reads	origin under 5% of 23 Tbps

System Design Case StudiesPage 11

Video Streaming Design Cheatsheet

1. Numbers (Video Streaming Platform)

Metric	Estimate
Users and plays	500M DAU x 5 videos = 2.5B plays/day
Play QPS	~29K avg, ~90K peak
Egress	2.5B x 100 MB = 250 PB/day (~23 Tbps)
Storage added	500K x 500 MB = 250 TB raw; with 6 renditions 500 TB/day
Read:write

System Design Case StudiesPage 12

Check yourself

The ladder is 240p 0.4, 360p 0.8, 480p 1.5, 720p 3 and 1080p 6 Mbps, with 4 s segments. Throughput falls from 4 Mbps to 1 Mbps while the buffer holds 8 s. Which rung should the next segment use, and what happens if the player picks 480p?

Pick 360p: it is the highest rung at or below 1 Mbps.
360p segment = 0.8 x 4 = 3.2 Mb, so it downloads in 3.2 s and the buffer grows by 0.8 s.
480p segment = 1.5 x 4 = 6 Mb, so it takes 6 s to download against 4 s of playback.
The buffer drains 2 s per segment, so it empties after about 4 segments and the player stalls.
Switch down fast: waiting to confirm the drop costs buffer.

A re-encoded rendition is written to the same path as before, /v/123/720p/seg5.m4s. Segments carry a 1-year TTL. Viewers keep seeing the old video. What is the bug and the fix?

Quantity	Math	Result
Plays	500M x 5	2.5B/day
Play QPS	2.5B / 86,400	~29K avg, 90K peak
Raw uploads	500K x 500 MB	250 TB/day
Stored (6 renditions)	about 2x raw	500 TB/day
Storage growth	500 TB x 365	~180 PB/year
Egress	2.5B x 100 MB avg	250 PB/day
Bandwidth	250 PB / 86,400 s	~23 Tbps avg
Read:write	2.5B plays / 500K uploads	~5000:1

Component	Decision	Why
Object store	Buy S3	11 nines durability, no ops
CDN	Buy CloudFront / Akamai	global POPs on day one
Transcoding	Build	cost and quality control (VMAF, per-title)
Recommendation	Build	core product differentiator
Own CDN	only above ~Tbps scale	Netflix Open Connect

uploads column	Holds
`upload_id`	PK, returned by POST /uploads
`user_id`, `size`	owner and declared bytes
`parts_done`	progress counter
`state`	INITIATED to READY or FAILED
`raw_object_key`, `created_at`	raw file location, start time

	HLS	DASH
Manifest	`.m3u8` text playlists	`.mpd` XML
Support	Native on Apple	Open standard, most other players
Codecs	Apple-defined set	Codec-agnostic
Segments	TS or fMP4	fMP4

Object	TTL	Rule
Segments `.m4s`	1 year	immutable, versioned path
Manifest, VOD	1 h	short, may be re-published
Manifest, live	2 s	playlist grows every segment
New release	pre-warm	push to edges by predicted popularity

Failure	Response
Edge POP down	DNS failover to the next POP
Origin overload	request coalescing + stale-if-error
One CDN degraded	multi-CDN, health-based traffic steering

Endpoint	Does	Notes
`GET /videos/{id}`	metadata + signed manifest URL	cached in Redis
`PATCH /videos/{id}`	edit title, tags, visibility	write, then invalidate
`GET /search?q=&page_token=`	ranked results	cursor pagination, no offset

	Display count	Billing count
Source	Stream pipeline	Raw log, batch job
Accuracy	Approximate	Exact
Lag	Seconds to minutes	Hours (offline)
Used for	Watch page, search	Creator payouts

Metric	Target
Event ingestion lag p99	under 10 s
Count drift vs batch truth	under 0.1%
Dedup ratio	tracked
Bot-filtered %	tracked

Store	Key	Holds
`user_embedding`	`user_id`	`vec`
`video_embedding`	`video_id`	`vec` (ANN index)
Feature store	`user_id`	watch history, genre affinity
Event log	`event_id`	impressions, clicks, watch time

Failure	Fallback
Model down	cached trending list per region
Feature store down	cached trending list per region
Cold-start user	popularity + onboarding picks

Axis	Option A	Option B	Pick
Storage vs compute	Pre-encode all renditions	On-demand encode	A for hot, B for long-tail
Quality vs cost	AV1: ~30% smaller, 5-10x slower encode	H.264 default	AV1 only for hot titles
Accuracy vs latency	Exact count	Windowed stream count	Stream to display, batch to bill

Design a Video Streaming Platform like YouTube

1. Scope and Functional Needs (Video Streaming Platform)

2. Non-Functional Targets

3. Back-of-Envelope Estimates

1. Two Planes, Two Paths

1. Responsibilities and Design (Video Streaming Platform)

1. Responsibilities and Design

1. What ABR Is and What the Manifest Lists

1. Responsibilities and Tiers

1. Responsibilities and Design

1. Responsibilities and Design

1. Job and Funnel

1. Scaling Levers (Video Streaming Platform)

1. Numbers (Video Streaming Platform)

2. Component Map and Async Flow

3. Build vs Buy

2. Flow and Data Model

3. Failures

4. Metrics

2. Flow and Data Model

3. Failures

4. Metrics

2. Player Algorithm and Example

3. HLS vs DASH

4. Common Mistakes

2. Request Flow and Cache Policy

3. Access Control

4. Failures and Metrics

2. Read and Edit Flow

3. Data Model and APIs

4. Failures and Metrics

2. Event Flow

3. Data Model

4. Failures and Metrics

2. Request Flow

3. Data and API

4. Failures and Metrics

2. Failure Handling

3. Trade-offs

4. Interview Answers

2. Upload and Transcode

3. Adaptive Bitrate (ABR)

4. Delivery and Counting