Last Minute Notes
  • Github
  • Docs
  • Learn
    @sayanstwt
Lessons
0/37 done
  • Queues & Streams
  • Real-Time Updates
  • REST, gRPC & GraphQL
  • Events & the Outbox
Learn/System Design/Communication

Queues & Streams

8 min Intermediate 6 steps

Decouple producers from consumers and absorb bursts.

User waits
for the worker
Spike
drops work
Coupling
tight
Web appEmail workeroverloaded200 emails
request response data
In this lessonStep 1 of 6
  1. The problem
  2. Buffer
  3. Partitions
  4. Delivery
  5. Failures
  6. Recap
Step 1 / 6problem

The slow work blocks the request

The web app sends 200 emails by calling the worker directly. The user waits, and a traffic spike overloads the worker.

Producer and consumer are tied to the same speed and the same uptime.

Key idea

Slow work on the request path hurts latency. A spike with no buffer becomes dropped work.

←→ step·R replay·Space pause

You'll learn

  • How a queue absorbs a spike
  • Partitions and consumer groups
  • At-least-once delivery and idempotent consumers
  • What a dead-letter queue is for
Read the full notesHLD Building Blocks, 11