Last Minute Notes
  • Github
  • Docs
  • Learn
    @sayanstwt
Lessons
0/37 done
  • How a Request Travels
  • Back-of-Envelope Math
  • Latency & Percentiles
  • From One Server to Many
  • Availability & the Nines
Learn/System Design/Foundations

Latency & Percentiles

6 min Beginner 7 steps

Why averages lie and the slowest 1% decides how fast you feel.

Requests
8
Typical
~60 ms
Slowest
720 ms
0ms200ms400ms600ms800msreq 144 msreq 250 msreq 356 msreq 460 msreq 564 msreq 671 msreq 778 msreq 8720 ms
normal slow / failed marker
In this lessonStep 1 of 7
  1. The problem
  2. Percentiles
  3. Fan-out
  4. Hedging
  5. Little’s law
  6. Throughput
  7. Recap
Step 1 / 7problem

The average lies

Eight requests: seven take well under 100 ms, one takes far longer.

The average lands somewhere no request actually was. It hides the slow tail, and users experience the tail.

Key idea

Never report latency as an average alone. Use percentiles.

←→ step·R replay·Space pause

You'll learn

  • Why averages hide what users feel
  • Read p50, p95, p99 and p99.9
  • Explain how fan-out amplifies the tail
  • Size in-flight work with Little’s law
  • Tell latency and throughput apart
Read the full notesSystem Design Fundamentals, 7