Content Delivery Networks

At 8:00 pm on the first night of its Diwali sale, an online store in Mumbai sends a notification to 20 million phones. Within a minute, shoppers in Guwahati, Chennai and Delhi open the app, and every one of them asks the Mumbai servers for the same 400 KB sale banner, the same scripts and the same product photos. The servers' network links fill in seconds. Pages that loaded in one second now take ten, and checkout, the one request that earns money, waits behind a million copies of a banner. Nothing about that banner is personal. It is one file, sent from one city, over and over.

This sale is the chapter's running example, and it has two problems. The first is distance. Light in optical fibre covers roughly 200 km every millisecond, about two thirds of its speed in a vacuum, and real cables wind through routers instead of running straight. A round trip from Guwahati to Mumbai takes tens of milliseconds; reaching a server on the US east coast takes about 200 ms.

The second is bytes. If 20 million shoppers each load the banner in the first hour, that is 8 TB. Delivery is priced per GB, roughly $0.05–0.10 at list prices (check current prices), so the banner alone is $400–800 of traffic, and for image and video products this egress usually costs more than storing the files.

A content delivery network (CDN) attacks both problems by keeping copies of files on servers close to users. It cannot make the bytes disappear, but it moves them off your own links. The next page measures what distance costs before the first byte even arrives.

A 200 ms round trip would not matter if a page needed only one. It needs several before the first byte of the reply arrives, and more for the images, scripts and fonts it pulls in. Ignoring DNS and the server's own work:

Distance is multiplied by every round trip before the shopper sees anything.

Newer protocols shrink the table, not the gap. HTTP/3 runs over QUIC, a transport on UDP that folds the connection and TLS handshakes into one round trip, so the first byte arrives after about 400 ms from far away or 20 ms from nearby. A lost packet also stalls only its own stream, which helps on patchy mobile networks.

Either way, a nearby server is about twenty times faster for anything it already holds, and no line of your code changes. Which raises the next question: how does a CDN put servers near everyone, and how does a phone find the right one?

A CDN runs edge servers: caching servers placed in many cities, often inside or next to large internet providers' networks. A site full of them in one city is a point of presence (PoP), and a large CDN has hundreds. Your own servers, which hold the real copy of everything, are the origin: for the store, its servers in Mumbai.

Shoppers must reach a nearby PoP without knowing it exists. Two methods do the steering, both met before: CNAMEs and location-based answers in DNS: How a Name Becomes an Address, and anycast in Load Balancers.

Many CDNs combine the two. Inside a PoP, a load balancer often uses consistent hashing on the URL, from Designing a Key-Value Store, so each file is cached on one or two servers instead of every one, which multiplies how many files the PoP can hold.

When a request reaches an edge, the edge looks in its cache. A hit is served at once. A miss sends the edge to the origin, and it keeps the file for next time. The share of requests served from cache is the hit ratio, the most important number on a CDN dashboard.

Each PoP fills its own cache, so the first shopper in each city pays for a miss and everyone after is served nearby. An expired copy is checked, not blindly downloaded again: the edge asks whether the file changed, and a 304 Not Modified reply carries only headers.

In the scene, files reached the edge only when someone asked. That is one of two ways to fill a CDN:

Pull trades one miss per PoP for almost no work; push trades work and storage fees for certainty. Most teams pull, and pre-warm before a known rush: an hour before 8 pm, a script (or the CDN's prefetch feature) requests the banner and the sale page's files through each region, so the rush finds them cached. Push wins when files are so large and so predictable that even one miss per PoP would hurt the origin.

Once files are at the edge, the edge must decide when two requests want the same copy. It stores each response under a cache key, by default the host name, the path and the query string. The rule for a good key is short: it holds everything that changes the response, and nothing else. Both directions of mistake are common:

• Too specific, and the hit ratio collapses. ?utm_source=whatsapp and ?utm_source=instagram make two copies of the same sale page, each missing separately. Strip tracking parameters, sort the rest so ?size=m&color=red matches ?color=red&size=m, and lower-case the host.
• Too broad, and users get the wrong thing. If the key ignores language, the Hindi sale page can reach a shopper who asked for Tamil.
• Cookies usually mean personal. By default, many CDNs will not cache a response that sets a cookie. Serve static files from a separate, cookie-free domain so their requests never carry the login cookie.

The two mistakes fail differently. A key that is too specific shows up on the dashboard as a falling hit ratio and a busy origin. A key that is too broad shows up as complaints, because it serves the wrong content quietly, which makes it the worse of the two.

Path and query are the easy part. The origin can also add request headers to the key, and that is where the costliest mistakes hide.

The origin adds a request header to the cache key with the Vary response header. Vary: Accept-Language tells every cache to keep a separate copy for each value of that header. On a header with a few values, that is exactly right; on a header with thousands, it quietly switches caching off. Here is how the common choices behave:

The pattern: vary only on a header the edge has reduced to a handful of values.

When mobile and desktop pages truly differ, have the edge sort requests into a small device class, such as mobile or desktop, and key on that. The same trick serves an A/B test: the edge assigns each shopper to group A or B and keys on the group, not on the cookie that carries it.

The key decides which copy a request gets. The next question is how long that copy may be served.

How long a copy stays fresh is its TTL (time to live). The origin sets it with response headers, so the caching policy lives next to your code. Three of the store's responses:

GET /static/app.3f9a2c.js
  -> Cache-Control: public, max-age=31536000, immutable

GET /sale/index.html
  -> Cache-Control: public, max-age=0, s-maxage=60, stale-while-revalidate=30, stale-if-error=86400

GET /api/cart
  -> Cache-Control: private, no-store

A year (31,536,000 s) for a file whose name changes with its content, a minute at the edge for the page that links to it, never for a shopper's cart.

s-maxage applies only to shared caches such as the CDN, and overrides max-age there; browsers ignore it. So browsers check the sale page every time, which the nearby edge answers with a cheap 304, while the edge keeps its own copy for 60 seconds. You can purge a CDN, but you cannot purge a browser's cache, so give browsers the shorter TTL.

The last two directives help when the origin struggles. stale-while-revalidate lets the edge serve the old copy for 30 more seconds while it fetches a fresh one in the background, so no shopper waits for the refresh. stale-if-error lets the edge keep serving the last good copy for a day if the origin is failing. Support varies by CDN, but where it exists, an origin outage shows slightly old pages instead of errors.

A year-long TTL raises an obvious question: how does a changed script ever reach shoppers? The answer is to never change a file under the same name. Each build puts a hash of the file's content into its name, so app.3f9a2c.js becomes app.4b1e7d.js when the code changes. A given name never changes, so a year-long TTL is safe. The HTML that references the files gets a short TTL, such as a minute, and a deploy reaches every shopper within a minute without a single purge.

The last step is why old files stay for days: a shopper who opened the page at 7:29 still asks for the old script, and deleting it would break their page mid-sale. Versioning also makes rollback instant, because the previous HTML still points at files that exist.

Versioned names make deploys cheap, but each new file still costs one miss in every PoP. The next page removes most of those.

With 50 PoPs, every new file and every expiry of the sale page can cost 50 separate misses, all landing on the origin at 8 pm together. An origin shield is a middle layer of cache: every edge sends its misses to one shield location, usually close to the origin, and only the shield talks to the origin. Fifty misses become one.

The shield helps most with the long tail: the store's 2 million product photos, each requested too rarely for every PoP to keep. Suppose edges hit 90% of requests and the shield hits 80% of what reaches it. The origin then sees 10% × 20% = 2% of requests instead of 10%: a fifth of the load, from one extra layer.

The price is one extra hop, paid only on misses: the edge-to-shield leg adds a few to tens of milliseconds. The shield also concentrates load, so it must be sized for every PoP's misses. And the design must say what happens if the shield location fails: edges usually fall back to the origin, so the origin must survive that for a while.

A shield also makes the riskiest CDN operation, the purge, far gentler, as the next page shows.

At 8:10 pm, the store notices a TV listed at ₹4,999 instead of ₹49,999. Checkout reads prices from the origin, so no order goes through at that price, but the product page is cached for 10 minutes and thousands of shoppers are seeing it. So the team purges: it asks the CDN to drop the page from every PoP and the shield. A purge reaches every PoP within seconds on some CDNs and minutes on others. The order of steps matters more than the purge itself.

Fix the origin before purging, or the edges refetch the mistake within seconds. The shield then turns the purge's 50 misses into one origin fetch, which matters most when a mass purge drops thousands of files at once.

Two refinements help. A soft purge marks copies stale instead of deleting them, so edges keep serving the old copy while one request revalidates it; use it for routine updates, and a hard purge for content that must not be seen again. A tag-based purge drops every response carrying a tag: tag each page showing the TV with product-8812, and one purge clears the product page, the sale listing and search results together.

Purges are for mistakes; most content should simply be cached or not. Ask two questions of every response: is it the same for everyone, and can it be a little old? If both answers are yes, it belongs on the CDN. Here is how the store's responses sort out:

Everything shared and slightly stale goes to the edge; everything personal or written goes through.

Three cases look cacheable but are not worth it. The deep long tail, such as a product page viewed once a month, almost never hits, so a cached copy buys little. The price at checkout must come from the origin, even if the product page showed a cached one. And some data must, by law or contract, stay out of third-party caches entirely.

The deal API row hides the most useful trick in the chapter, and it gets the next page.

The sale's lightning deal page shows a countdown and the stock left, and a million shoppers keep it open, each polling every 2 seconds: 500,000 requests a second for one response. Cache it for just 1 second at each of 50 PoPs, and each PoP asks the origin about once a second. The origin sees about 50 requests a second instead of 500,000. This is micro-caching.

It works only if, when a copy expires, the edge sends one request to the origin and makes everyone else wait for that answer. This is request collapsing. Without it, the hundreds of requests that arrive in the instant after expiry all miss together and reach the origin at once, every second.

The price is a stock count up to a second old. That is fine for display, because the order itself checks stock at the origin with a conditional update, the "fresh read is not a lock" rule from Abstractions That Hide the Network. The trick flips for anything personal, and for anything whose staleness costs money on its own.

Micro-caching serves shared live data. Checkout is neither shared nor cacheable, yet a CDN still speeds it up.

A shopper in Guwahati checking out against a far-away origin pays about three long round trips before the first byte. Behind a CDN, the same uncacheable request is faster, through dynamic content acceleration:

• TLS ends at the edge. The handshakes run with the nearby edge, and the edge forwards the request over a warm, already open connection to the origin. From 200 ms away, roughly 600 ms becomes a little over 200 ms.
• Better paths. The CDN carries traffic over its own well-connected network instead of whatever route the public internet picks.
• Small logic at the edge. Redirects, A/B group assignment or a token check can run on the edge, so some requests never travel to the origin.

So even an origin that serves only personal pages gains from a CDN in front. The edge in front of everything also has one more job: keeping attackers out.

At 8:00 pm the store's real shoppers arrive together with bots: scripts that grab deals and floods meant to knock the site over. Because the CDN sits in front of everything, it is the first line of defence:

• DDoS absorption. With anycast, an attack flood spreads across hundreds of PoPs with enormous combined capacity, instead of landing on a few origin servers.
• Filtering. Firewall rules and rate limits at the edge drop bad requests before they cost you anything; Rate Limiters covers the algorithms.
• TLS at the edge. The CDN holds your certificate and encrypts close to users. Keep the leg from the edge to the origin encrypted too.
• Signed URLs. For paid videos or private files, the origin issues links with a signature and an expiry time, and the edge refuses any link without a valid one.

Most CDN trouble comes from a few repeat mistakes. Here they are with what users see and the fix:

The second row does the most damage, because it leaks private data instead of only slowing pages. Treat is this response personal? as a question every endpoint answers before launch, not after a shopper reports seeing a stranger's address.

As soon as images, video or a spread-out audience appear in the requirements, bring in the CDN and say exactly what goes on it.