Design a Microblog

9:52 pm: India hit the last ball of a World Cup final for six. Kabir, a former fast bowler turned commentator with 30 million followers, posts two words: "WHAT A FINISH." Two million fans post in the same minute. On a first-draft design, Kabir’s post alone means 30 million timeline writes, queued ahead of everyone else’s. Ravi, who follows Kabir and 299 other accounts, refreshes and sees nothing new. He refreshes again, and each refresh asks the database to rebuild his timeline from scratch. The trophy is lifted before his timeline catches up. This chapter builds the design that keeps up.

The product: a microblog where people post short public messages, follow other accounts and read a home timeline of the newest posts from everyone they follow. Plan for about 200 million daily users. A few accounts, like Kabir’s, have tens of millions of followers. You are designing the posting path, the follow graph and the home timeline.

Before any numbers, pin down what the product does, because the hard part only shows once the scope is clear.

A microblog is a service for short public posts. People publish a few lines of text, follow other accounts and read a home timeline: the recent posts of everyone they follow, newest first. Following is one-way, so Ravi can follow Kabir without Kabir following him back. That is how one account collects 30 M followers while most have a few dozen.

Writing a post is easy: one row per post. Reading is the hard part. Every time someone opens the app, the system must show a fresh timeline stitched from the posts of hundreds of accounts, in a fraction of a second, for hundreds of millions of people. The whole design follows from making that read cheap.

The scope below keeps the timeline at the centre and leaves out features that would each bring their own write path or read path.

Everything on the right can be added later without changing how the home timeline is built, which is why it can wait. The next question is how fast, how fresh and how safe the timeline must be.

Scope says what the product does; non-functional requirements say how well, and they decide the architecture. Each one below gets a number or a clear rule.

• Fast reads. The home timeline returns in under 200 ms at the 99th percentile (p99), measured at the server: 99 of every 100 requests are faster than that.
• Read-heavy. People scroll far more than they post: about 20 timeline loads for every post written, and each load shows 20 posts.
• Available rather than strictly consistent. If Kabir’s post reaches a follower 5 seconds late, nobody is harmed. If timelines stop loading during the final, everybody notices. So timelines are eventually consistent: every copy catches up after a short delay.
• Durable. Once the app says "posted", the post is never lost.
• Read your own writes. Ravi always sees his own new post at once, even if his followers see it a few seconds later.

These assumptions feed every estimate in the chapter.

Two rows matter most: 200 followers on average, and 30 M for the largest accounts.

With the assumptions set, the question is which number decides the design. Start with the cheap one. 100 M posts a day is about 1,160 posts a second (100 M ÷ 86,400), the arithmetic from Back-of-the-Envelope Estimates. One well-tuned database could absorb that alone.

The expensive number hides one step further. If every post id is copied into each follower’s timeline, one post becomes 200 writes.

The follower count, not the post rate, sets the size of the system. Fan-out turns 1,160 posts a second into more than a million timeline inserts at the peak, and it says those inserts must run in the background through a queue, because nobody should wait for 200 writes.

The other estimates are quick, and each answers a different question.

Storage is ordinary; the timeline inserts and the timeline cache are what need design. The 1 KB per post is generous on purpose: 280 Devanagari characters take up to 840 bytes in UTF-8, at 3 bytes each, and the post also carries ids, a timestamp and flags.

The estimates say reads dominate, so the API should keep both paths simple. A handful of calls covers every requirement, and the server learns who is calling from the auth token, never from a user id in the body.

POST   /v1/posts                 {"text": "What a finish", "media_ids": ["m_91"]}
       Idempotency-Key: 6f1c2a   -> 201 {"post_id": "1834092", "created_at": "..."}
DELETE /v1/posts/{post_id}       -> 204
PUT    /v1/follows/{user_id}     -> 204   (follow; repeating it is harmless)
DELETE /v1/follows/{user_id}     -> 204   (unfollow)
GET    /v1/timeline/home?limit=20&cursor=1834092  -> 200 {"posts": [...], "next_cursor": "1829117"}
GET    /v1/users/{user_id}/posts?limit=20&cursor=...
PUT    /v1/posts/{post_id}/likes/me              -> 204
GET    /v1/search?q=last+over&cursor=...

Images are uploaded to the blob store first; the post refers to them by media id.

Idempotency key. Phones on crowded stadium networks time out and retry. The app attaches a random key to each new post; if the server has seen that key in the last 24 hours, it returns the original result instead of creating a second, identical post. Without it, every timeout during the final becomes a duplicate.

Cursors, not page numbers. New posts arrive at the top while Ravi scrolls, so "page 3" points somewhere different every second and he sees the same posts again. A cursor that says "posts older than id 1829117" is not moved by new posts at the top, so the next page carries on exactly where the last one ended. This works because the timeline is sorted by id; Design a News Feed shows what changes when the list is ranked instead.

Each call reads a different shape of data. The next page decides where each shape lives.

Four kinds of data sit behind those calls, and each is read in its own way. Pick storage from the access pattern, as the Databases chapter recommends, not from habit.

The table maps each question the system asks to the layout that answers it from one partition.

The follow graph is stored twice, once by follower and once by followee, because fan-out asks "who follows Ravi?" while timeline rebuilds ask "whom does Ravi follow?". Two writes per follow is a small price for answering each from a single partition.

With the data placed, the system splits into two paths. The write path saves a post and hands a job to a queue. The read path serves a home timeline from a cache. Fan-out workers join the two: they read the author’s follower list and push the new post id into each follower’s cached timeline. Follow one of Ravi’s posts through it.

Each component has one job:

• Post service: assigns a time-ordered id and writes the post plus the outbox row on its shard, the outbox pattern from Message Queues.
• Fan-out queue and workers: a durable queue, so a burst of posts becomes a backlog instead of an outage.
• Timeline cache: one capped list of post ids per user, in memory, sharded by user id.
• Timeline service: reads that list, then hydrates it: turns ids into full posts with author names, counts and image links, read from a post cache.
• Post stream: a topic, as in Publish-Subscribe, that search, trending and notifications consume at their own pace.

The expensive work happens at posting time, where nobody waits for it. That choice is called fan-out on write. Is it always the right one?

Think of a newspaper. Home delivery does the work before anyone wakes up, once per subscriber. A newsstand does nothing until a reader walks over and collects each paper. Timelines can be built either way.

Fan-out on write, also called push, copies each new post id into every follower’s timeline when the post is written. Fan-out on read, or pull, does nothing at posting time: when Ravi opens the app, it fetches the recent posts of all 300 accounts he follows and merges them. Set side by side, the two fail in opposite places.

Push breaks on huge accounts; pull breaks on read traffic. The daily totals settle the default. Push costs 100 M posts × 200 followers = 20 B small inserts, all in the background. Pull costs 2 B loads × 300 accounts = 600 B lookups, each inside a request a person is waiting for. Push does about 30 times less work, and none of it while anyone waits.

So push is the right default, but not for Kabir. The next page splits the two.

Push works for Ravi and fails for Kabir, so treat the two kinds of account differently. Pick a threshold, say 100,000 followers. Posts from accounts below it are pushed as before, so no single post costs more than about 100,000 inserts. Posts from accounts above it, the high-fan-out accounts, are saved but never fanned out. They are pulled when a reader opens the app and merged into the pushed list.

Pulling sounds like the expensive path we just rejected, but here it is cheap. A typical reader follows only a handful of such accounts, and their recent posts are read by millions, so they almost always sit in the post cache. Pulling five of them costs a few milliseconds.

Pushing Kabir’s post would have queued 30 M inserts ahead of every other post; skipping it costs each of his readers one extra cache read. The hybrid moves the work for a few huge accounts from posting time to reading time, where caching makes it nearly free because millions of readers share the same few posts.

The merge is where the two halves meet, so it must be fast and stay correct when they overlap. Here is the whole read path in a few lines.

def home_timeline(user_id, limit=20, before=None):
    ids = timeline_cache.range(user_id, before=before, count=limit)  # pushed ids
    for star in following_high_fanout(user_id):                      # usually 0 to 10
        ids += user_posts.recent(star, before=before, count=limit)   # hot in cache
    ids = sorted(set(ids), reverse=True)[:limit]                     # newest first
    return hydrate(ids)       # full posts; drops deleted posts and blocked authors

Time-ordered ids make the merge a sort of a few hundred integers, and set() removes duplicates.

One load of Ravi’s timeline, with 295 accounts pushed and 5 pulled, breaks down like this.

The load uses under a tenth of the 200 ms budget. Three races need care:

• Crossing the threshold. An account that goes viral can pass 100,000 followers in a day. Re-check the flag every few hours, not on every follow; a post both pushed and pulled during the switch still shows once, because the merge removes duplicate ids.
• The author’s own post. Fan-out is asynchronous, so the post service writes the id into the author’s list before replying, or Ravi thinks his post failed and posts again.
• Late jobs. A delayed job can deliver a 10:00:01 post after a 10:00:05 one. The list is kept sorted by id, so the late id lands in its place.

The threshold is a dial, not a constant: set it so a typical reader merges no more than about 10 pulled accounts.

Below the threshold, push stays the default, and the jobs themselves need shaping. One post can still mean up to 100,000 inserts. As a single job, that keeps one worker busy for about five seconds at 20,000 inserts a second. If the worker crashes near the end, the whole job runs again, and every post queued behind it waits.

So split big jobs into chunks. The first job for a post reads the author’s follower list a page at a time and queues one chunk job per 1,000 followers, each naming the post id and its slice of the list. A 90,000-follower post becomes 90 chunks that many workers run at once, finishing in well under a second. A crash repeats one chunk, and that is harmless: inserting an id that a sorted list already holds changes nothing. Ravi’s 200-follower post stays a single job.

Two habits cut the work further:

• Skip inactive users. Do not push to accounts that have not opened the app in 30 days. When one returns, build the timeline once with fan-out on read, then put them back on the push list.
• Push to the recently active first. During a backlog, order chunks so people who opened the app in the last hour get the post first; the rest can wait a minute.

Size the tier from the estimate scene: at about 20,000 batched inserts a second per worker, the 230,000 per second average needs about 12 workers and the final’s 1.2 M peak about 60. Scale between the two with the queue depth, and let the queue absorb anything above 60 as a few seconds of lag rather than errors. Every insert lands in the timeline cache, which the next page sizes.

Every insert from the workers lands in the timeline cache, so it must hold every active user’s list and take a million writes a second at the peak. How big is it?

Each user’s timeline is a list of post ids, newest first, capped at 800 entries. A push is two cheap steps: insert the id, then trim back to 800. If someone scrolls past the end, the timeline service falls back to fan-out on read for older pages.

The raw ids are small: 300 M monthly actives × 800 × 8 bytes ≈ 1.9 TB. How they are stored decides the real size. A compact packed encoding roughly doubles it with keys and slack, to about 4 TB. A general-purpose sorted structure with pointers per entry can cost 5 to 10 times the raw bytes, which would mean 10 to 20 TB, so the encoding is a cost decision, not a detail. The choices below assume the compact one.

Memory, not traffic, sets the server count. At the peak, each of 60 primaries takes about 20,000 inserts and 2,000 timeline reads a second (1.2 M and 115,000 spread across them), far below the roughly 100,000 operations a second one cache node handles.

The cache holds only ids, so every load turns ids back into posts. That step, hydration, is also where deletes and blocks take effect.

Hydration sends one batched multi-get, a single request for many keys, to the post cache, which holds post objects by id, then does the same for author profiles. A popular post appears in millions of timelines, and Kabir’s is read by millions of people, but each is cached once, so the post cache’s hit ratio is very high. Hot posts also sit in each server’s local memory for a few seconds, so millions of readers do not all hit one cache key.

Storing ids instead of whole posts is what makes changes cheap. If timelines held copies of posts, deleting a post from a 90,000-follower account would mean rewriting 90,000 lists, and every like would change them all again. With ids, a delete marks the post deleted in the post store and the post cache, and hydration drops any id whose post is deleted or whose author you have blocked or muted. The stale id falls off the end of each list as newer posts push it past 800.

An unfollow uses the same filter until the old ids age out. A new follow copies that account’s recent posts into your list in the background, so the next load already includes them.

With every part in place, the next question is what breaks first as load grows, and what users notice when it does.

Each part of the design fails in its own way. The table lists each failure, what users see, and how the design copes.

In almost every row the timeline keeps loading: failures show up as lateness or a slower first load, rarely as errors. That is the eventual consistency chosen in the requirements, paying off.

Every choice in this design bought something and paid for it. Push spends memory and background work to make reads cheap, and accepts a few seconds of staleness. The hybrid adds a second way to build a timeline, more code and more tests, in exchange for never melting on a post like Kabir’s.

Running in two regions adds one more trade. Each region keeps its own timeline caches and fan-out workers, and posts are copied between regions asynchronously, so a post written in Mumbai reaches a follower served from another region a second or two later. Copying synchronously would make every post wait for a cross-region round trip, about 200 ms to another continent, to buy freshness nobody asked for.

Three signals show whether the design is healthy:

Image egress from the CDN dominates the bill. One post in ten carries an image, so 23,000 loads a second × 20 posts pull about 46,000 images a second: at 100 KB for a feed-size copy, roughly 12 PB a month, about $600,000 at $0.05 per GB (check current prices). Memory for about 120 cache servers comes next. Post text, at 37 TB a year, is cheap.

Optional: three features that grow from the post stream once timelines work. None of them sits on the posting path, so if one falls behind, posting and timelines carry on unaffected.

Search. An indexer reads new posts from the stream, splits the text into terms and adds the post id to each term’s list in an inverted index, as in Distributed Search. People mostly search for what is happening now, so the index is split by time: a small, hot index for the last few days and larger, colder ones behind it. A query reads the hot index first and goes further back only when it needs more results.

Trending. Count hashtags and phrases in short windows, say 5 minutes, per region. A topic trends when its count is far above its usual count for that hour, not merely when it is large. A greeting hashtag that is big every morning is not news; a player’s name jumping from 200 to 40,000 mentions in the last over is. The pipeline has five steps:

Likes and reposts. Kabir’s post can collect thousands of likes a second, all aimed at one number. Use Sharded Counters: each like increments one of N shard counters, and readers see a cached sum refreshed every few seconds. Store the like itself as a row keyed by post and user, and add to the count only when that row is new, so liking twice changes nothing. A repost is a small post that points to the original, so it fans out like any other post and hydration shows the original inside it.

Here is the whole answer on the interview clock: about 40 minutes of design once introductions are done.

Pick the deep dive from the interviewer’s questions. Famous accounts, write load or late posts: take the hybrid and chunked jobs. Latency, memory or deletes: take the timeline cache and hydration. Search or trending: spend five minutes on the post stream consumers, then come back.

Interviewers often change the problem halfway to see whether the design bends or breaks. Answer each one before opening it.