📘 System Design Fundamentals

This page is the shared base for both HLD and LLD. It teaches how to approach a design problem, how to size a system with quick math, and the vocabulary every later page assumes: scalability, availability, latency, consistency. If you only have an hour, read sections 2, 4 and 8.

Table of Contents

1. What System Design Is
2. The Seven-Step Framework
3. Requirements
4. Back-of-Envelope Estimation
5. Scaling Stages
6. Availability and Reliability
7. Latency, Throughput and Percentiles
8. CAP and PACELC
9. Consistency Models
10. Core Trade-offs
11. Beginner Questions
12. Cheat Sheet

1. What System Design Is

System design is choosing components, data flows and failure behavior so a product meets its requirements at a given scale, cost and reliability. There is no single right answer. There are answers that fit the requirements and answers that do not.

Two flavors appear in interviews:

A third flavor is growing fast: AI system design (LLM serving, RAG, agents). It is HLD with new components, covered in AI System Design.

What interviewers actually score, regardless of flavor:

• Problem framing: do you ask before you build?
• Structure: do you move through the problem in an order you can defend?
• Depth: can you go one level deeper on at least two components?
• Trade-offs: do you name what you give up with every choice?
• Failure thinking: what breaks, and what does the user see when it does?
• Communication: can the interviewer follow and steer you?

2. The Seven-Step Framework

Use the same skeleton every time. It keeps you from jumping to boxes and arrows before you know what the system must do.

Rules of thumb:

• Say your assumption out loud, then write it down.
• Start with a design that works for one server, then remove the bottlenecks one at a time.
• Do not name a technology until you can say what property you need from it.
• Prefer boring technology (Postgres, Redis, Kafka, S3) unless a requirement rules it out.
• End every component with "and what happens when it fails".

3. Requirements

3.1 Functional Requirements

What the system does, phrased as user actions. Keep them to a short list you can finish.

Example, URL shortener:

1. Given a long URL, return a short URL.
2. Given a short URL, redirect to the long URL.
3. Optional: custom alias, expiry, click analytics.

Say what is out of scope. "I will skip user accounts and abuse detection unless you want them."

3.2 Non-Functional Requirements

How well the system does it. These drive the architecture far more than the features.

A useful trick: for each requirement ask "what would break my design if this were 10x?"

3.3 Identify the Core Access Patterns

Before drawing a database, list the queries. The data model follows the queries, not the other way round.

4. Back-of-Envelope Estimation

The goal is not accuracy. The goal is to know whether you need one machine or one thousand, and whether the bottleneck is CPU, memory, disk or network.

4.1 Powers of Two and Time

Shortcuts:

• 1 million requests per day is about 12 requests per second.
• 100 million per day is about 1,200 per second.
• 1 billion per day is about 12,000 per second.
• Peak traffic is commonly 2x to 5x the average. Use 3x unless told otherwise.

4.2 Latency Numbers

Exact values drift with hardware, but the orders of magnitude are stable. These are rounded, modern-ish figures.

Take-aways that shape designs:

• Memory is roughly 1000x faster than SSD, and SSD roughly 100x faster than a cross-region call.
• A network call is expensive compared to a function call, so chatty microservices add up.
• Sequential disk access is far cheaper than random access. This is why logs (Kafka, LSM trees) are fast.
• Cross-region round trips dominate any synchronous multi-region design.

4.3 Worked Example: Photo Sharing App

Assumptions

• 100 million daily active users.
• Each user uploads 0.1 photos per day and views 50 photos per day.
• Average uploaded photo 2 MB, average thumbnail served 200 KB.
• Keep photos forever, replication factor 3.

Write path

Uploads per day  = 100M x 0.1        = 10M
Upload QPS avg   = 10M / 86,400      = about 116
Upload QPS peak  = 116 x 3           = about 350

Read path

Views per day    = 100M x 50         = 5 billion
View QPS avg     = 5B / 86,400       = about 58,000
View QPS peak    = 58k x 3           = about 175,000
Read:write ratio = 5B : 10M          = 500 : 1

Storage

New data per day = 10M x 2 MB        = 20 TB
Per year         = 20 TB x 365       = 7.3 PB
With 3 replicas  = about 22 PB per year

Bandwidth (egress)

5B views x 200 KB = 1 PB per day
1 PB / 86,400 s   = about 11.6 GB/s = about 93 Gbps average

Decisions the numbers force

• Reads outnumber writes 500 to 1, so cache aggressively and serve images from a CDN. The origin cannot carry 93 Gbps cheaply.
• Storage is petabytes per year, so use object storage (S3 class) with tiering, not a database.
• 350 write QPS is small. The upload path is not the hard part. Metadata and feed generation are.
• Metadata (photo id, owner, URL) is tiny compared to blobs. Keep it in a database and the blobs in object storage.

4.4 Server Count

Peak QPS               = 175,000
One app server handles = 5,000 QPS of simple requests (assume)
Servers needed         = 175,000 / 5,000 = 35
Add N+2 redundancy     = about 40 to 50 servers

If the estimate says 3 servers, do not draw a 200 node cluster. If it says 3,000, sharding and a serious cache tier are unavoidable.

4.5 Estimation Template

Common mistakes:

• Forgetting the peak factor.
• Forgetting replication in storage numbers.
• Doing 10 minutes of arithmetic that does not change any decision.
• Mixing bits and bytes. Network is quoted in bits per second, storage in bytes.

5. Scaling Stages

Most real systems, and most good interview answers, grow through the same stages. Name the stage you are at and what triggers the next one.

5.1 Vertical vs Horizontal Scaling

A single modern database server with NVMe and hundreds of GB of RAM handles far more than people assume. Do not shard early. Scale up, add replicas and caching first, and shard when writes or data size force it.

5.2 Stateless vs Stateful

A stateless service keeps no per-user state between requests, so any instance can serve any request. That makes scaling out and replacing instances trivial.

To make a service stateless:

• Store sessions in a shared store (Redis) or in a signed token (JWT).
• Store uploads in object storage, not on local disk.
• Keep caches that must be consistent outside the process, or accept per-instance staleness.

State does not disappear. It moves to the data tier, which is where you concentrate the hard problems: replication, sharding, consistency.

6. Availability and Reliability

6.1 Definitions

6.2 The Nines

Every extra nine costs roughly an order of magnitude more effort. Ask what the business needs. An internal report tool does not need five nines.

6.3 Availability Math

Components in series (all must work):

A_total = A1 x A2 x A3
Three services at 99.9% each = 0.999^3 = 99.7%

Components in parallel (any one is enough):

A_total = 1 - (1 - A)^n
Two replicas at 99.9% each = 1 - 0.001^2 = 99.9999%

Consequences:

• A long synchronous call chain lowers availability. Ten services at 99.9% give about 99.0%.
• Redundancy raises availability only if failures are independent. Two replicas in one rack, one AZ or one region share fate.
• The load balancer, DNS and any shared dependency become the new ceiling.

6.4 SLI, SLO, SLA

Details and burn-rate alerting are in Reliability and Operations.

6.5 Redundancy Patterns

7. Latency, Throughput and Percentiles

Latency is how long one request takes. Throughput is how many requests per unit time the system completes. They are related but not the same: batching raises throughput and often raises latency.

7.1 Use Percentiles, Not Averages

An average hides the slow tail. Users experience the tail.

7.2 Tail Latency at Scale

If a page needs 100 backend calls and each has a 1% chance of being slow, the chance that at least one is slow is 1 - 0.99^100, about 63%. Fan-out amplifies the tail.

Mitigations:

• Timeouts and deadlines that propagate down the call chain.
• Hedged requests: send a second request after the p95 time and use the first response.
• Cap fan-out, cache, or precompute.
• Load shedding and bulkheads so one slow dependency does not drain shared threads.

7.3 Little's Law

L = lambda x W
L      = average number of requests in the system
lambda = arrival rate (requests per second)
W      = average time each request spends in the system

If 1,000 requests per second each take 200 ms, about 200 requests are in flight at any time. Size thread pools and connection pools with this, and note that when W grows during an incident, L grows too, which is how overload spirals.

8. CAP and PACELC

8.1 CAP

In a distributed data store, when a network partition occurs, you must choose between:

• Consistency (C): every read sees the latest write (linearizability), or returns an error.
• Availability (A): every request to a live node gets a non-error response, possibly stale.

Partitions are not optional in a real network, so the real choice is CP or AP during a partition. Outside a partition you can have both.

Common misunderstandings:

• CAP does not say "pick two of three" in normal operation.
• It is a statement about behavior during a partition, and about linearizability specifically.
• Many systems are tunable per request (for example Cassandra quorum levels, DynamoDB strongly consistent reads).
• "CA" is not a real category for a system that spans a network.

8.2 PACELC

PACELC extends CAP: if there is a Partition, choose A or C; Else, choose Latency or Consistency. It captures that even without partitions, replication forces a trade-off between latency and consistency.

Use the table as a rough guide. Behavior depends on configuration, so say "it depends on the read and write concern" if asked.

8.3 Picking C or A by Domain

9. Consistency Models

From strongest to weakest.

9.1 Making Read-Your-Writes Work with Replicas

• Route the user's reads to the primary for a short window after they write.
• Track the write's log position and only read from replicas that have caught up.
• Pin the session to one replica so reads are at least monotonic.

9.2 ACID and BASE

Note that the "C" in ACID (application invariants) is not the "C" in CAP (linearizability). Isolation levels and their anomalies are in Distributed Systems.

10. Core Trade-offs

Almost every design decision is one of these. Name the trade-off aloud when you make the choice.

11. Beginner Questions

Q1. What is the difference between scalability and performance? Performance is how fast the system is for one request or at a given load. Scalability is how well it keeps its performance as load grows when you add resources. A system can be fast for one user and not scale, or scale but be slow per request.

Q2. Vertical or horizontal scaling first? Vertical first, because it is the simplest and modern machines are large. Move to horizontal for redundancy or when one machine cannot cope. Design the app tier stateless from day one so horizontal scaling stays cheap.

Q3. Why is availability of chained services lower than each service? Availability multiplies in series. Three 99.9% services in a synchronous chain give about 99.7%. Reduce chain length, add caching or fallbacks, and make non-critical calls optional.

Q4. What does "eventually consistent" mean, and when is it acceptable? If writes stop, all replicas converge to the same value, but reads in the meantime may be stale. Acceptable where staleness is harmless and availability or latency matters more: feeds, counters, recommendations, search.

Q5. Explain CAP in two sentences. When a network partition happens, a distributed store must choose between rejecting some requests to stay consistent or answering with possibly stale data to stay available. When there is no partition it can offer both, though PACELC says it still trades latency against consistency.

Q6. Why use percentiles instead of the average? The average hides the slow tail that real users hit, especially with fan-out where one slow call slows the whole page. p95, p99 and p99.9 describe user experience better and drive SLOs.

Q7. How do you decide the number of servers? Estimate peak QPS, measure or assume per-server capacity, divide, then add headroom for failures and deploys, typically N+2. Validate with load tests.

Q8. What is a single point of failure and how do you remove one? Any component whose failure takes the system down. Add redundancy (replicas, multiple AZs), remove shared fate (independent failure domains), and test failover regularly. Watch for hidden ones: DNS, the load balancer, a config service, a shared database.

Q9. Why is state the hard part of scaling? Stateless services scale by cloning. State must be replicated, partitioned and kept consistent, which is where CAP, replication lag and sharding complexity live.

Q10. What do you clarify first in any design question? Users and scale, the read to write ratio, latency and availability targets, consistency needs, what is in and out of scope. Then write the numbers down and let them steer the design.

12. Cheat Sheet

1 day = 86,400 s ~ 10^5 s        1M/day ~ 12 QPS        peak ~ 3x average
1 KB ~ 10^3   1 MB ~ 10^6   1 GB ~ 10^9   1 TB ~ 10^12   1 PB ~ 10^15 (bytes)
memory 100 ns | SSD 100 us | DC round trip 0.5 ms | cross-continent 100 ms
99.9% = 8.8 h/yr | 99.99% = 53 min/yr | 99.999% = 5 min/yr
series: multiply | parallel: 1 - (1-A)^n
Little: L = lambda x W
CAP: during a partition pick C or A. PACELC: else pick L or C.

Next: Building Blocks for the HLD components, or LLD Fundamentals for the class-design side.