Design a Q&A Site

At 9:40 pm on a weekday, a popular open-source charting library ships version 5.0, and by 10 pm builds are failing in offices and hostels from Bengaluru to Noida. Thousands of developers paste the same error into a search engine and land on one page: question 42, "Build fails after chartkit 5.0 update". Yesterday it had eight views an hour; tonight it has a thousand a second. If every view runs four database queries and adds 1 to a single view-count row, the database stalls and the one page everyone needs times out. This chapter keeps that page fast, its votes honest and its counts cheap.

You are building a Q&A site for programmers: people ask questions, others answer, and the community votes the best answers to the top. The site serves about 100 million page views a day, and most visitors arrive from a web search, read one page and leave without signing in. Question 42 is our running example, from a quiet page to its viral night.

Start with what the site must do, then with how well it must do it. Question 42 shows the shape: one asker, a few answerers, a few hundred voters, and a crowd of readers who never post anything.

A Q&A site stores questions, the answers under them and votes on both. A post's score is its upvotes minus its downvotes, and the score decides which answer a reader sees first. Most visitors search for an error message, read the top answer and leave. This is a read-heavy product, and the question page is the product.

The table splits each feature area into what we build today and what waits:

The scope is small on purpose; the difficulty lives in the non-functional requirements, each with a number the design can be held to:

• Fast pages: a question page loads in under 200 ms for 99% of readers, most of whom are not signed in.
• Correct votes: one vote per user per post, never counted twice, and voters see their own vote at once.
• Fresh enough: other readers may see a score that is a minute old, but a new answer should appear within seconds.
• Durable and available: a posted answer is never lost, and reading keeps working even when posting is degraded.

How much traffic do those promises have to survive? The next page puts numbers on it.

Requirements say what to build; numbers say where the effort goes. Assume 100 million page views a day, about 1 in 10 from signed-in users, 50,000 new questions and 100,000 answers a day, and 1 million votes a day. Most posts are 1 to 3 KB of text. Evenings run about 3 times the average.

Here is the working in one place:

Reads outnumber writes about 100 to 1, so every read must be cheap. The read rate itself is modest, which points to caching whole pages rather than to a giant database. Ten years of text fits on one large server; even with votes, edit history and indexes it is a few terabytes, so partitioning, if it comes, is for growth and easier operations, not for today's size.

The danger is concentration. Question 42 on its viral night takes about 1,000 views a second, almost a third of the whole site's peak, on one page. Its page and its view count are where the design must not break.

With the load known, the API decides which calls can be cached and which must reach the database. A short list covers the product; the interesting one is voting.

POST /v1/questions                 { title, body, tags[] }  -> 201 { questionId }
GET  /v1/questions/42?sort=votes                           -> 200 { question, answers[] }
GET  /v1/questions/42/my-votes                             -> 200 { votes: { postId: 1 | -1 } }
POST /v1/questions/42/view                                 -> 202   (sent by the page itself)
POST /v1/questions/42/answers     { body }                 -> 201 { answerId }
PUT  /v1/posts/{postId}/vote       { value: 1 | -1 | 0 }    -> 200 { myVote }
POST /v1/questions/42/accept      { answerId }             -> 200
GET  /v1/tags/{tag}/questions?sort=newest&cursor=c9         -> 200 { questions[], nextCursor }
GET  /v1/search?q=chartkit+5.0+build+fails&cursor=c9        -> 200 { results[], nextCursor }

Anonymous readers need no session; every write except the view ping needs a signed-in user.

Voting is a PUT that sets your vote, not a POST that adds one. A double tap that fires the request twice, or a phone retrying after a timeout, repeats the same request with the same result: your vote is +1. Undoing a vote is the same call with value 0. With a POST /upvote that adds a vote on every call, each retry would count again unless the server caught it.

my-votes is the small personal part of a page: which posts this user has voted on. Keeping it apart from the shared question is what lets the shared part be cached for everyone. The view ping exists for the same reason: a page served from a cache never reaches our servers, so the page reports its own view.

Notice that the vote call names only the post, not its question. That looks harmless now; the page on growing the database shows when it stops being harmless.

The calls fix the questions the store must answer. The entities are few and the relationships simple, so a relational database fits well: questions have answers, posts have votes, questions have tags.

Five details carry the design:

• The votes key enforces one vote per user per post. Questions and answers share one id space, so post_id can point at either, and a second row for the same post and user is rejected by the database itself.
• Scores are running totals, not sums of vote rows. Adding up vote rows on every page build would be far too slow, so post_scores keeps one row per post, changed in the same transaction as the vote.
• Views have their own table. Almost every question needs one row (shard 0); the shard column lets a hot question spread over more rows if batching is not enough.
• question_tags has one row per tag of a question, indexed by tag and date, so "newest questions in chartkit" is one index range read.
• question_id is in every child row, so a question, its answers, votes and counters can all live on the same shard later.

Edits append a row to a post_revisions table before updating the current body, so edit history and rolling back vandalism are simple reads.

Now put the boxes together. Signed-out readers, about 90% of views, get the page from a CDN edge that keeps it for up to a minute. Signed-in readers and every CDN miss reach the web servers, which check a page cache of rendered questions; on a miss they read the question from its database shard, build the page and store it. A vote commits the vote row and the new score in one transaction, then deletes the page-cache copy.

The vote changed two things at different speeds, on purpose. The person who cares most, the voter, saw it immediately; the crowd saw it within a minute, and in exchange the crowd cost our servers about one page build a minute instead of a thousand reads a second.

Think of a newspaper. It prints one edition for every reader and does not print a copy per subscriber; the address label is stuck on afterwards. A question page works the same way: nearly every reader of question 42 sees exactly the same page, so cache the whole rendered page, keyed by question id and sort order.

For signed-out readers, let the CDN keep the page with Cache-Control: public, s-maxage=60, max-age=0, as in Content Delivery Networks: edges keep it for 60 seconds, while browsers check every time and get a cheap answer from the nearby edge. The edge skips its copy for requests that carry a session cookie. Signed-in users are about 10% of views, and they are the ones who post, vote and edit, so they need fresh pages.

The numbers show why this matters. Question 42 at 1,000 views a second sends about 900 a second to the edges. Each edge location fetches the page at most once a minute, and an origin shield, a middle cache layer, folds those fetches into about one a minute at our servers.

Signed-in users still share the cached page. Do not cache a page per user: personal extras, such as their own votes and an edit button on their own posts, come from the small my-votes call, and the browser combines the two. A per-user cache would hold ten copies of question 42 for ten readers and hit almost never.

The page cache need not hold every question. About 2 million questions are viewed in a day, and at about 50 KB a page that is 100 GB, a few cache servers, with least-recently-used eviction keeping the busy pages. A cached page is only useful if it is fresh enough, so how long should each copy live, and what deletes it?

Two caches hold question 42, and they need different rules because they serve different readers. The table sets one rule per layer:

Short TTLs belong at the edge; the page cache keeps a page until it changes. Why not a 60-second TTL in the page cache too? Most questions are read less than once a minute, so their copies would expire between readers and nearly every view would miss.

Kept for a day and deleted only on change, a page misses on its first view of the day and after each change. That is at most 2 million first views plus about 1.2 million changes (1 million votes, 150,000 posts and some edits), roughly 3 million builds out of 100 million views. The hit ratio is about 97%, and at a 3× evening peak that means about 110 builds a second, inside a budget of 175 a second, which is 5% of 3,500. Each build is a few queries on one shard.

Two traps remain. A rebuild right after a vote must read from the primary database, not a replica: a replica a second behind would put the old score back in the cache for a day, the stale refill described in Distributed Caches. And when question 42's entry is deleted while hundreds of signed-in readers want it, mark it stale instead: one request rebuilds while the others get the old copy for a moment (request coalescing).

A purge at the CDN reaches every edge in seconds on most networks, so a new answer shows within seconds, and within a minute at worst, when the TTL runs out.

Caching handles the readers; now the writers. A vote is rare but must be exact. Each vote does three things: record who voted, change the score, and tell other systems, and the order matters.

The dangerous moment is a double tap that sends the same vote to two web servers at once. Both read the old vote before either has written, so both believe the change is +1.

Correctness came from the database, not from careful servers. Locking the old vote in step 1 cannot help, because when there is no vote yet there is no row to lock; the primary key on (question, post, user) is what catches the race. The vote row and the score row share question 42's shard, so steps 1 to 4 run as one local transaction, and the losing copy costs only a retry that adds nothing.

A vote that must be exact gets a transaction. A view needs something else entirely, as the next page shows.

Votes and views are both counters, so why treat them differently? Because they sit on opposite sides of the limit of a single row.

Every update to one row waits for the lock held by the update before it, so one row caps how fast one number can change: about 1,000 updates a second when each commit waits for a local disk, and about 200 when it waits for a replica in another zone, as Sharded Counters shows. Votes stay far below that: 12 a second site-wide, and perhaps 50 a second on question 42 at its worst. One score row is plenty.

Views are the hot counter. Question 42 takes about 1,000 view pings a second, right at the cap of one row. The two standard fixes compare like this for that one question:

Batching wins for views. Each web server keeps a small map of question id to count and every 5 seconds writes one views = views + 437 per question. With 40 web servers that is about 8 writes a second for question 42 however viral it gets, and views tolerate both the lag and the rare loss. Shard the row only where batching is impossible, for example if pings land on short-lived functions with no memory between requests, or if every view must be durable.

One more rule keeps the count honest: count a visitor once per question every 15 minutes. A small cache key per visitor and question, set only if absent, filters out refreshes and simple bots before a ping reaches the batch.

With votes and views settled, the page must decide which answer comes first. The default is simple: the accepted answer first, then by score, newest first among ties.

Question 42 shows the flaw. Its accepted answer, written for chartkit 4 three years ago, has 900 votes and is now wrong. A new answer that fixes the 5.0 build has 40 votes after one night. Sorted by score, the wrong answer stays on top for months, because old answers had years to collect votes. The site offers a few orders:

Trending is the useful default for pages like question 42: the old answer's 900 votes, spread over three years, now weigh only about 50, while the fix's 40 fresh votes weigh 40 and keep growing, so it passes within days. Some sites also stop pinning the accepted answer when another answer clearly outscores it, because the asker may never come back to change the choice.

Ranking stays cheap. A question usually has a handful of answers and rarely more than a few dozen, all on one shard, so the web server sorts them while building the page. No separate ranking service is needed, and each sort order is simply another cached copy of the page.

Readers do not only arrive at a question directly; they browse tags and search, which the next page covers.

Question pages are one way in; tag listings and search are the others, and each needs its own index.

Tag listings. Pages like "newest in chartkit" or "highest voted in android" read from question_tags, indexed by (tag, created_at) and (tag, score); its score is a copy refreshed every few minutes, which is fine for sorting. A question with three tags writes three rows when posted. That small fan-out on write makes every listing one index range read. The first page of a busy tag is cached for 10 to 30 seconds.

Search. An inverted index, as in Distributed Search, covers titles, bodies, tags and answers, fed from the event queue a few seconds behind the database. Ranking mixes text relevance with signals this product has in plenty: score, accepted answers, views and recency.

The same index powers a valuable extra on question 42's night: dozens of people start to ask the same question. Showing likely duplicates while someone types a title points them to question 42 before they post, which saves answerers and moderators hours. Code needs care when text is split into terms: a search for NullPointerException should match the whole word, and "null pointer" should ideally match it too.

Every one of these paths reads from the database somewhere. How far can one database go before it must be split?

Grow the database in steps, and let the numbers decide each one:

At these numbers, one primary with replicas could run the site for years, so do not partition early. Partitioning pays off when the site grows many times over, or when chores such as backups, schema changes, index rebuilds and failover get slow on one big database. When you do, question id is the natural key: almost every request is about one question, and one question's data is small.

Here is the catch from the API page. PUT /v1/posts/{postId}/vote names only the post. Once data is split by question id, the vote service must know the question before it can find the shard. Either put the question id in the URL, which the page already knows, or build every post id with its question's shard number inside, so the id alone routes the request. Either way, the shard checks that the post belongs to that question.

After the split, each request finds its data like this:

Walk each failure from the reader's side: what they would notice, and what keeps the site up. Question 42's viral night is a good test for each row:

The pattern is the same in every row: reading degrades last. The CDN, the page cache and the replicas each keep pages flowing while the part behind them recovers, and only posting, the rare action, waits.

Vote retries are missing from the table on purpose: thanks to PUT semantics and the votes primary key, a repeated vote is not a failure at all. The best failure handling is a design where the failure cannot do harm.

Each choice so far bought something and cost something. These are the ones worth naming:

• Stale scores for cheap reads: signed-out readers may see a score a minute old, and in return question 42 costs about one build a minute.
• Batched views: up to 5 seconds of lag and a rare small loss, in exchange for about 8 writes a second instead of 1,000.
• Fan-out on write for tags: three index rows per question cost little and make every listing one fast read.
• Soft delete: keeps evidence and allows undo, but hidden posts must be filtered from every read path and from search.

Serving pages dominates the bill. 100 million page views of about 50 KB is roughly 5 TB a day out of the CDN before compression, about 150 TB a month; at $0.05 to $0.10 per GB of egress (check current prices), compression and long-lived static files matter more than any database choice. All the text ever posted is about 1 TB.

The cheapest trade-off of all is the one the requirements made: deciding that a minute-old score is fine. Every cache in this design leans on that one sentence.

Optional: how an open site keeps spam, vote fraud and abuse in check once the core works.

An open site that anyone can post to attracts spam within days, and a viral page like question 42 attracts it within minutes, because spam answers there are seen by thousands. No single check stops it, so use layers, cheapest first:

Vote fraud needs its own check. Extra accounts that upvote their owner, or a group that votes each other up, leave a pattern: many votes between the same few accounts in a short time. A nightly job finds these patterns and reverses the votes, which is easier and fairer than blocking each one as it happens.

Reputation unlocks privileges, so compute it with care. A worker reads vote events from the queue and appends a line to the user's reputation ledger, such as +10 when their answer is upvoted, keyed by the vote's event id so a redelivered event is skipped. The profile shows the ledger's sum, cached on the user row. Reversed votes append opposite entries instead of editing history, so every point can be explained.

A 45-minute round leaves about 40 minutes of design once introductions and your own questions are done. Here is how to spend them on a Q&A site.

Pick page caching if they ask about latency, signed-out traffic or a viral question. Pick votes and views if they ask about correctness, double votes or hot counters. If they push on growth, go to partitioning by question id and how a vote finds its shard.

Interviewers change one thing to see whether your design bends or breaks. Try each question before opening the answer.