Rate Limiting and Idempotency

7 pages · ~5 min✓ Reviewed

REST API DesignPage 1

Rate Limiting Mental Model

1. What It Does

Rate limiting caps requests per client per time window
Excess requests are rejected or delayed
Protects the server and enforces fairness

2. Key and Knobs

Limit key	Shares one budget across
API key	one client app
User ID	one account
IP address	everyone behind one NAT
Route	one endpoint

Sustained rate: requests per second, long-run
Burst size: how many can arrive at once

3. Where It Runs

	Server-side	Client-side
Protects	the service	your own quota
Goal	reject excess	stay under provider limit

bash

INCR rl:user42:minute_28
EXPIRE rl:user42:minute_28 60

shared atomic counter in Redis (or one Lua script)

Per-node counters

3 nodes each allowing 100/min really allow 300/min. Share one atomic counter.

REST API DesignPage 2

Token Bucket, Fixed and Sliding Window

1. Token Bucket

Holds up to capacity tokens, refills at rate r per second
Each request spends 1 token
Empty bucket → reject
Bursts up to capacity are allowed

Refill is lazy: no timer thread
State per key: tokens, last_refill_time

python

tokens = min(capacity, tokens + elapsed * rate)
last_refill_time = now

run on each request

2. Fixed Window

Count per calendar window (e.g. per minute)
Reset counter at the boundary
Cheap: one counter per key

Boundary flaw

Full limit at the end of one window plus full limit at the start of the next gives up to 2x the limit in a short span.

3. Sliding Window

Variant	Stores	Accuracy
Log	every request timestamp	exact, memory-heavy
Counter	two window counters	approximate, cheap

Counter weights the previous window: current + previous * (1 - elapsed fraction)

REST API DesignPage 3

429 Too Many Requests and Retry-After

1. Status and Headers

429 = standard rate-limit rejection; 503 = general overload
Neither is a payload bug: don't edit the request
RateLimit-* / X-RateLimit-*: informational, not universal; read docs

Retry-After	Form
`30`	delay-seconds
`Wed, 21 Oct 2026 07:28:00 GMT`	HTTP-date

2. Client Rule

429
Retry-After?
Wait at least that
Retry

Parse both forms; prefer it over your own backoff

python

from email.utils import parsedate_to_datetime as p
from datetime import datetime, timezone
def wait_s(v):
    if v.isdigit(): return int(v)
    d = p(v) - datetime.now(timezone.utc)
    return max(0, d.total_seconds())

3. Mistake and Interview Point

Tight retry loop

Retrying a 429 immediately worsens overload and can extend the ban.

Retry-After vs backoff? Retry-After wins; backoff is the fallback
429 vs 503? Rate limit vs general overload

REST API DesignPage 4

Retries and Idempotency Keys

1. What to Retry

Retry transient failures: timeout, 429, 502, 503, 504
Never retry 4xx like 400, 401, 404: they fail again

2. Backoff and Jitter

Doubling: wait 1s, 2s, 4s, ...
Cap: a maximum wait per attempt
Max attempts: stop after N tries
Jitter: randomise the delay so clients do not retry in lockstep (retry storm)

python

import random, uuid
key = str(uuid.uuid4())  # once per operation
headers = {"Idempotency-Key": key}
wait = random.uniform(0, min(30, 2 ** n))

fragment: n = attempt number, cap 30s

3. Idempotency Keys

Method	Effect of a retry
GET, PUT, DELETE	safe: same state
POST	unsafe: duplicate, e.g. double charge

ClientPOST + key
Serverstores key + first result
Repeatsame stored result

Key rule

Reuse the same key on every retry. A new key only for a new operation.

REST API DesignPage 5

Webhooks Basics

1. What a Webhook Is

An HTTP POST the provider sends to your registered URL
Event happens → provider pushes; you stop polling
Delivery is at-least-once and unordered

2. Handling an Event

Verify HMACraw body
Return 2xxfast
Queue jobasync work
Dedupeevent ID

Provider signs the raw bytes with a shared secret (HMAC-SHA256)
Never verify re-serialised JSON
Slow or failed response → provider retries
Store the event ID; skip seen ones to stay idempotent

python

import hmac, hashlib
mac = hmac.new(secret, raw_body,
               hashlib.sha256).hexdigest()
ok = hmac.compare_digest(mac, sig_header)

recompute, then compare in constant time

3. Common Mistake

Trusting the payload

Skipping the signature or timestamp check allows spoofed and replayed events.

REST API DesignPage 6

Rate Limiting and Retries Cheat Sheet

1. Algorithms

Algorithm	Behaviour	Cost
Token bucket	bursts, steady average	2 values
Fixed window	2x boundary burst	1 counter
Sliding counter	smooth, approximate	2 counters
Sliding log	exact	memory per request

2. Reject and retry

Server: 429 + Retry-After (seconds or HTTP-date) + RateLimit-* headers
Retry timeouts, 429, 502, 503, 504; never other 4xx, 500 or 501
Honour Retry-After, else exponential backoff with cap and jitter

Send
Retryable?429, 502-504
WaitRetry-After or backoff
Attempts left?else give up

3. Idempotency and webhooks

One Idempotency-Key per logical POST, reused on every retry
Server returns the stored result for repeats
Webhook: verify HMAC on the raw body, return 2xx fast, queue work
Dedupe by event ID

Mistake

Instant 429 retries worsen overload; a new key per retry duplicates the POST.

Why jitter? Clients stop retrying in lockstep.
Why the raw body? Re-serialised JSON changes the signed bytes.

REST API DesignPage 7

Check yourself

Fixed window, limit 100/min. A client sends 100 requests at 12:00:59 and 100 more at 12:01:00. How many get through?

All 200: they land in two different windows.
That is 2x the limit in about a second, the boundary flaw.

A POST /charge times out. The client retries with a freshly generated Idempotency-Key each time. What goes wrong?

The server sees new keys, so it may charge twice.
Create the key once per operation and reuse it on every retry.

A webhook handler credits an account on every delivery. Event evt_7 arrives twice. What is the bug?

Delivery is at-least-once, so duplicates happen.
Store the event ID and skip ones already processed.