Design a Group Chat App

11:59 pm on New Year’s Eve. In the Fest crew group, Asha asks “Who’s bringing the cake?” Ravi replies “Me!” and Meera replies “Not me.” On Kabir’s phone, “Me!” appears above the question. On Ravi’s laptop, Meera’s reply shows twice. And Asha’s next message, ticked as sent, never reaches anyone: the server holding it restarted under the midnight load. Three bugs, one cause. The app had no single agreed order for the chat, and no durable record of who was still owed each message.

You are building a chat app for phones and laptops. People message one to one and in groups of up to 500, share photos, and see when messages are delivered and read. A message must arrive instantly when both people are online and must never be lost when one of them is not. About 200 million people use it every day.

Before sizing anything, pin down what the app does. Conversations are either one-to-one or groups of up to 500 members: a family group, a hostel wing, or the Fest crew from the opening. Here is the scope for this design:

Two properties matter more than anything else in that list. Messages in a conversation must appear in the same order for everyone, and a message the server has acknowledged must never be lost, even if the recipient’s phone stays off for a week. The rest of the chapter is built around those two promises.

The next question is how fast, how safe and how private the app must be, and how much traffic it carries.

Chat is judged on speed and trust. Turn both into rules you can check:

• Low latency. With both people online, a message arrives in under 500 ms end to end, and usually far less within one country.
• Durable and ordered. No acknowledged message is ever lost, and each conversation has one agreed order.
• At-least-once delivery, shown once. When an acknowledgement is lost, the sender cannot tell a lost message from a lost ack, so it resends, and a message can arrive twice. Phones recognise what they already hold and show each message once.
• Highly available. A chat server crash should cost its users a few seconds of reconnecting, never their messages.
• Private. Only the people in a conversation can read it, using end-to-end encryption, covered at a high level later.

The estimates on the next page rest on these assumptions:

Two of these drive the design: the share of users online at once sets the number of open connections, and recipients per message turn each send into several deliveries.

With the assumptions set, size the two things chat servers actually feel: deliveries per second and open connections.

Plan for the peak, because at midnight on New Year’s Eve everybody greets everybody at once. That 5× is the low end of the estimates chapter’s festival range, so the design also needs a plan for shedding extras when traffic goes higher. The rest of the sizing follows from the same assumptions:

Two numbers shape the design: 1.4 M deliveries a second and 60 M open sockets. Storage is large but ordinary, and the bytes are in attachments, which belong in a blob store.

Sixty million open connections raise the first question: how does a server push a message to a phone that did not ask for it? Plain HTTP cannot, because the server only answers when the phone asks. There are three common ways around that:

WebSockets win for chat because an idle connection costs a heartbeat, not a stream of empty replies; long polling is the fallback where networks block them.

A WebSocket starts as an HTTPS request that asks to upgrade the connection; after the server answers 101 Switching Protocols, the same TLS-encrypted TCP connection carries small frames both ways. The load balancer picks a chat server when the connection opens, usually the least loaded one (see the load balancer chapter), and the phone stays there until the connection drops. Both sides send a heartbeat, such as a ping every 30 seconds, because a phone that loses signal sends no goodbye: two missed heartbeats tell the server within about a minute that the connection is dead.

With 600 chat servers, Asha’s message usually reaches a different server from the one holding Ravi’s connection. A session directory answers “which chat server holds user X right now?”. It is an in-memory key-value store, sharded by user id, that maps each connected device to a chat server. A server writes the entry on connect and deletes it on disconnect. A crashed server cannot clean up, so entries expire after a couple of minutes unless refreshed, and a dead server’s entries simply run out.

Next, decide what travels over the connection and where it is kept. Most traffic is small WebSocket frames; a few HTTP calls handle the chat list, history, new groups and attachments:

GET  /v1/connect   Upgrade: websocket   -> 101 Switching Protocols, then frames both ways
Asha -> {"type": "send", "conv": "c_88", "client_msg_id": "a1f3", "body": "<ciphertext>"}
Asha <- {"type": "ack", "client_msg_id": "a1f3", "seq": 39}
Ravi <- {"type": "message", "conv": "c_88", "seq": 39, "from": "u_asha", "body": "<ciphertext>"}
Ravi -> {"type": "receipt", "conv": "c_88", "upto_seq": 39, "kind": "delivered"}
GET  /v1/inbox?limit=50   -> 200 [{"conv": "c_88", "last_seq": 40, "unread": 2}, ...]   (newest first)
GET  /v1/conversations/c_88/messages?after_seq=39&limit=100   -> 200 {"messages": [...]}
POST /v1/conversations  {"members": ["u_ravi", "u_meera"], "title": "Fest crew"}  -> 201 {"conv": "c_91"}
POST /v1/attachments    -> 200 {"put_url": "...", "key": "att_7q2"}   (upload straight to the blob store)

Asha -> is Asha’s phone sending; Ravi <- is the server sending to Ravi’s phone. Bodies are encrypted on the phone; the socket is authenticated when it opens.

Two ids do the work. The client_msg_id is made by the phone, so a resend after a timeout is recognised as the same message and answered with the original seq. The seq is assigned by the server from one counter per conversation, and it fixes the order everyone sees. Four tables hold the state:

Messages are written once and read in order within a conversation, so a wide-column store from the databases chapter fits: partition by conversation, sort by seq, keep three replicas. The key also carries a bucket number, seq ÷ 100,000, so no partition grows past about 20 MB of text. Attachments go to a blob store through signed upload URLs and are served by a CDN.

user_inbox is each user’s chat list, sorted by last activity, so a phone loads its 50 most recent chats with one range read. The unread badge is last_seq − last_read_seq, two numbers the design already keeps. A stored counter would cost a write per member per message and drift whenever a retry delivered twice.

With the data in place, put the pieces together and follow one message from Asha to Ravi. Phones hold WebSockets to chat servers behind a load balancer. For each conversation, one chat server acts as its owner: it numbers each message, writes it to the message store and to a fan-out log, and only then acknowledges it. Fan-out workers deliver from the log, and a push queue wakes phones whose app is closed. The parts, in one list:

Follow one message, first while Ravi is online and then while his app is closed.

Three rules carry the design. Store before you acknowledge, so once Asha sees “sent”, no crash can lose the message. Deliver to a socket when you can, send a push when you cannot, and on reconnect let the phone sync from its cursor, the last seq it holds in each chat.

The picture hides two hard questions: who hands out the seq, and what happens when that server or a fan-out worker dies mid-message. The deep dives take them in turn.

Start with the simpler half. If Ravi and Meera both reply in the same second, how does every member see the two replies in the same order? Phone clocks cannot decide it: they drift by seconds, and some are simply set wrong. Here are three Fest crew messages whose clocks disagree:

Sorted by phone clocks, Ravi’s reply would appear above the message it answers, and Asha’s would come last. Instead, the server gives each conversation its own counter: every accepted message takes the next seq, and every device sorts by it. A reply is sent only after its sender has seen the original, so it always gets a higher seq. Ravi’s and Meera’s messages could land in either order; what matters is that everyone sees the same one.

Sequence numbers also reveal gaps. A phone that holds 41 and receives 44 knows it is missing 42 and 43, so it fetches them before showing 44. Time-ordered ids like those in the unique IDs chapter cannot do this: they are unique and roughly sorted, but nothing tells you an id in between is missing.

A counter is easy on one machine. The next page asks what happens when the machine holding it fails.

A counter needs exactly one owner at a time. Conversations are split into shards by consistent hashing, and each shard’s owner, a chat server, holds a lease on it: a right from a coordination service that expires unless renewed, such as 10 seconds renewed every 3 (see the coordination chapter). The owner keeps each chat’s next seq in memory, so numbering costs no database round trip; even at the 465,000-a-second peak, each owner numbers under 1,000 sends a second.

When an owner dies, its lease runs out and another server takes the shard. The dangerous case is an owner that only freezes, in a long pause, and wakes still sure it owns the shard.

The fence is in the store, not the lease. Every message is written with insert-if-absent on (conv_id, seq), a conditional write that wide-column stores offer for a few extra milliseconds, so no seq can ever hold two messages. Like the coordination chapter’s fencing token, it lets the store decide who wins. If the old owner’s late write reaches a still-free seq first, it simply wins and the new owner re-reads; the old owner stops at its next failed renewal, within about 3 seconds.

The fence keeps every seq unique. Two smaller cracks remain, and each could hide or duplicate a message: a write whose outcome is unknown, and a resend that reaches a new owner.

A write that times out. As the abstractions chapter showed, a timeout does not say whether the write happened. If the owner simply skipped that seq and phones stopped waiting for it, a write that landed later would sit where no phone looks, and Asha would have been told “sent”. So the owner settles every failed seq: it writes a tombstone, an empty placeholder, at the same seq with the same insert-if-absent, retried until it gets an answer. Either the tombstone lands, and nothing can be written there ever again, or the original write had landed after all, and the owner acknowledges the message with that seq.

Phones never skip a missing seq. A phone that holds 50 and receives 52 fetches 51 and waits until it is either a message or a tombstone: usually milliseconds, at worst one takeover. It skips only tombstones, so an acknowledged message can never hide behind a phone’s cursor.

A resend that reaches a new owner. Say the old owner stored Asha’s message as seq 60, then died before the ack reached her. Her phone resends with the same client_msg_id. A new owner that knew only the highest seq would store the message again as 61, and every member would see it twice. So before numbering a chat’s first message after a takeover, the new owner reads that chat’s recent tail, the last few hundred messages with their client_msg_ids. It answers a resend of a stored message with the original seq, and tombstones any gap it finds in the tail, settling writes the old owner left open.

Reading the tail costs one range read per chat, a few milliseconds, paid once per takeover. Put together, a takeover pauses sends in that shard’s chats for about one lease, roughly 10 seconds, and never loses or doubles a message. Ordering is settled; the next question is how one stored message reaches all 500 members.

A stored message must reach every member even if a server crashes halfway, so no chat server fans out from memory. After the store write, the owner appends the message to a durable fan-out log, partitioned by conversation as in the message queue chapter, and only then acknowledges the sender. If the owner dies between the two, the sender resends, and the new owner finds the stored message by its client_msg_id and logs it then.

The log turns a crash into a repeat, never a loss. Workers commit their offset, the log’s bookmark, only after forwarding, so a replacement redoes unfinished work and phones drop any seq they already hold. Delivery is at least once; display is exactly once.

For each message a fan-out worker looks up the members; moves the chat to the top of each member’s chat list; asks the session directory which servers hold their devices; forwards one copy per server; and queues a push for anyone offline. One worker per partition keeps each chat in log order. How much work one message causes depends on group size:

A 500-member group spread over hundreds of servers costs about 500 deliveries and 500 chat-list updates per message, which is why groups have a size cap.

Fan-out finds no live session for some members. What do their phones get, and how do they catch up without missing or repeating anything?

For each such member, the worker puts a small job on a push queue: recipient, conversation and seq. Push workers call the phone platform’s push service. The push carries no readable text: with end-to-end encryption it holds either the still-encrypted message for the phone to decrypt or just a signal that something is waiting. One push per chat is enough until the chat is opened, and muted chats get none.

Push is best effort: it can arrive late, out of order or not at all, so it is only a nudge. The message store, not the push, is the source of truth. When the app opens, it runs reconnect sync:

The cost is small. Ravi’s phone, off for a weekend, might hold 2,000 new messages across 30 chats: about 400 KB of text in a few dozen range reads, well under a second on a mobile network.

Each device keeps its own cursors, so a laptop that was off for a week catches up from exactly where it stopped. Read state belongs to the user: reading on the laptop sends “read up to seq 87”, which clears the badge on the phone. The same sync makes a server crash cheap: its 100,000 phones reconnect elsewhere and each fetches only its own gap.

Synced phones report back: delivered, then read. How do you record those reports for 500 members without multiplying the work? Each receipt has a trigger and a storage form:

Store receipts as watermarks, “everything up to seq N”, not one row per message per member. In a 500-member group, per-message rows would add about 1,000 rows, delivered and read, for every message; watermarks stay at two numbers per member however long the chat runs. Forwarding every receipt to the sender would double the fan-out, so in big groups the server updates the watermarks, tells the sender once everyone has the message, and builds the “read by” list only when someone opens it.

Presence, the online dot and “last seen”, is chattier than it looks. If 60 M online users each changed state 10 times an evening and every change went to 200 contacts, that would be about 8 M updates a second over four hours, more than five times the peak message deliveries. So presence is read, not pushed: the directory knows who is connected, the server saves “last seen” when a user’s last device disconnects, and a phone asks only about the people in the chat it has open.

Typing indicators go only to members with the chat open, are never stored, and are the first thing dropped under load. Next, put the whole design under stress: what fails, and what users notice.

With the design complete, walk through each moving part and what a user would notice if it failed:

Every row ends in a reconnect, a pause or a hidden repeat, never a lost acknowledged message, because the store and the log hold everything and chat servers hold nothing they cannot rebuild.

Each guarantee in that table has a price. Most trade-offs come down to what to drop first under pressure: messages never; read receipts can be delayed; typing indicators and online status can be skipped; history sync for old chats can be paced.

The second trade-off is where state lives. Chat servers hold only connections and, for the shards they own, each chat’s next seq, and both are cheap to rebuild. The price is a conditional store write and a log append on every send, a few milliseconds that buy crash safety.

Going global adds a third. Give each conversation a home region, usually where most members are, that owns its shard. Members elsewhere connect to nearby chat servers, which forward to the home region, so a message between Delhi and London pays one long round trip of about 100 to 150 ms, still inside the 500 ms budget. Copy each message to the standby region before the ack: about 30 ms more between two Indian regions buys zero loss when a region dies.

Retention. Text is small, about 580 TB a year before replicas, so keep it as long as the chat exists. Attachments are 50 times bigger and rarely opened after a few weeks, so move them to a colder, cheaper tier after about 30 days. Three signals show trouble early:

What dominates the bill: attachments. Storing about 80 TB of new photos and files a day and serving each to every recipient usually costs more than the text and the chat servers together, which is why old media moves to a colder tier.

Optional: what changes when even the servers must not read the messages?

With end-to-end encryption, messages are encrypted on the sender’s phone and decrypted only on the recipients’ devices. Private keys never leave those devices, so servers store and forward ciphertext they cannot read. The design barely changes, which is why the API showed <ciphertext> bodies from the start. A direct chat works like this, in outline:

Groups are harder: a separate copy for every member’s device would mean hundreds of copies per message. A common approach gives each sender their own key for the group, often called a sender key, shared once with every member’s devices over their one-to-one encrypted channels. Each message is then encrypted once, and the server fans out the same ciphertext. When someone leaves, the others make new keys and share them with everyone but that person; a new member gets only the current keys, so they cannot read older messages.

A new device, such as a laptop, makes its own key pair and is approved from a device the user already has. Attachments are encrypted on the phone with a fresh key that travels inside the message, so the blob store and CDN hold only unreadable bytes. The protection has costs elsewhere:

• The server cannot search content, so search runs on the device.
• Spam detection relies on metadata, such as sending rates, and on user reports.
• Backups must be encrypted with a key only the user holds, or they undo the protection.

A 45-minute round leaves about 40 minutes of design. Here is this chapter on that clock:

The owner, the fence and the log in the middle rows are what make this design hold up. Pick ordering as the deep dive if they ask about two people sending at once or a server dying mid-send; pick fan-out if they ask how one message reaches 500 members, or what offline members get.

Interviewers change one thing and watch you adapt. Answer each before you open it.