Last Minute Notes
  • Github
  • Docs
  • Learn
    @sayanstwt
Lessons
0/37 done
  • DNS Resolution
  • Content Delivery Networks
  • Load Balancing
  • Gateways, Proxies & Meshes
  • Caching
  • Rate Limiting
Learn/System Design/Traffic & Edge

Rate Limiting

8 min Intermediate 6 steps

Token bucket, sliding window and where to enforce a limit.

Capacity
~10 / s
Spike
31 / s
Result
overload
PER SECOND0123454553164
current done skipped
In this lessonStep 1 of 6
  1. The problem
  2. Token bucket
  3. Fixed window
  4. Sliding window
  5. Distributed
  6. Recap
Step 1 / 6problem

Protect the service from overload

A client or a bug sends far more requests than the service can handle. It slows down for everyone, not just the noisy caller.

Rate limiting says: you may do N requests per time window, the rest get 429 Too Many Requests.

Key idea

Rate limiting protects against overload, abuse and noisy neighbours. The choice of algorithm decides how bursts are handled.

←→ step·R replay·Space pause

You'll learn

  • How a token bucket allows bursts but caps the rate
  • Why fixed windows leak a double burst at the edge
  • The sliding window counter formula
  • How to share a limit across servers
Read the full notesDistributed Systems, 9