📘 HLD Building Blocks

Every system design answer is a composition of a small set of parts. This page explains each part: what it is for, how it works, its failure modes, and when to choose it. It is written from beginner level up, and each section ends with the trade-off an interviewer will probe.

Table of Contents

1. The Request Path
2. DNS
3. CDN
4. Load Balancers
5. API Gateway, Reverse Proxy, Service Mesh
6. Caching
7. Databases
8. Indexing and Storage Engines
9. Replication
10. Sharding and Consistent Hashing
11. Queues and Streams
12. Object Storage
13. Search
14. Real-Time Communication
15. APIs and Protocols
16. Choosing the Right Block
17. Questions and Answers

1. The Request Path

Trace one request end to end. Most designs are variations on this path.

At every hop ask: what does this add (latency, failure mode, cost) and what does it buy (scale, safety, speed)?

2. DNS

DNS maps names to IP addresses. It is the first dependency of almost every request, which makes it a hidden single point of failure.

TTL controls how long resolvers cache an answer. Short TTL gives fast failover and more DNS load. Long TTL is cheap and slow to change.

DNS-based traffic steering (Route 53, Cloudflare, NS1) offers:

• Weighted: send 5% of traffic to a canary.
• Latency-based: send users to the closest healthy region.
• Geolocation: send by country, for compliance.
• Failover: health checks flip to a standby.

Anycast announces the same IP from many locations, and the network routes each client to the nearest one. CDNs and public DNS use it.

Trade-off: DNS failover is bounded by TTLs and by resolvers that ignore them. Use it for coarse regional failover, not for per-request routing. The October 2025 AWS incident began with an empty DNS record for a regional service endpoint, a reminder that DNS is code you depend on. See Reliability and Operations.

3. CDN

A CDN caches content at edge locations close to users. It cuts latency, offloads the origin, and absorbs traffic spikes and some attacks.

Static assets are the obvious win. Also cache API responses that are the same for many users (product pages, public feeds) with short TTLs.

Trade-off: caching dynamic or personalized content risks serving one user's data to another. Vary on the right headers and never cache responses that carry Set-Cookie or private data without care.

4. Load Balancers

A load balancer spreads requests across servers, removes unhealthy ones, and gives clients one stable address.

4.1 Algorithms

4.2 Operational Details

• Health checks: active (probe an endpoint) and passive (watch errors). Separate liveness (restart me) from readiness (send me traffic).
• Connection draining: stop new requests, let in-flight ones finish before removal on deploys.
• Sticky sessions: pin a user to a server via cookie. Avoid by making services stateless. If unavoidable, plan for the server dying.
• TLS termination: decrypt at the LB, then re-encrypt to backends if zero-trust requires it.
• The LB is a SPOF: run a pair with a virtual IP, or use a managed multi-AZ LB, or anycast.
• Global load balancing: DNS or anycast picks a region, a regional LB picks a server.

5. API Gateway, Reverse Proxy, Service Mesh

Guidance:

• Put cross-cutting concerns (auth, throttling, logging) in the gateway so services stay small.
• Do not put business logic in the gateway.
• A mesh helps when you have many services and want uniform policy. It adds operational weight, so skip it for a handful of services.

6. Caching

A cache stores the result of expensive work so the next request is cheap. It is the highest leverage tool for read-heavy systems, and the biggest source of subtle bugs.

6.1 Where to Cache

Closer to the user is faster and harder to invalidate.

6.2 Read and Write Patterns

For writes, delete the key instead of updating it. Updating races with concurrent readers and writers and can leave stale values. Even with delete, a slow reader can repopulate an old value. A short TTL is the safety net.

6.3 Eviction

An LRU cache is a classic LLD problem, see Machine Coding 1.

6.4 The Classic Cache Problems

// single-flight sketch: one DB load per key at a time
const inflight = new Map();

async function getUser(id) {
  const key = `user:${id}`;
  const hit = await cache.get(key);
  if (hit) return hit;

  if (!inflight.has(key)) {
    const p = db.loadUser(id)
      .then(async (u) => { await cache.set(key, u, { ttl: 60 + Math.random() * 30 }); return u; })
      .finally(() => inflight.delete(key));
    inflight.set(key, p);
  }
  return inflight.get(key);
}

6.5 Redis, Valkey, Memcached

Licensing note (2024 to 2025):

• In March 2024 Redis moved from BSD to source-available licenses (RSAL and SSPL) starting with 7.4.
• The Linux Foundation forked the last BSD release as Valkey, backed by AWS, Google, Oracle and others. Managed services from cloud providers now offer Valkey.
• In May 2025 Redis 8 added the AGPLv3 open source license as an option.
• For interviews the API is the same for what you need. Say "Redis or Valkey" and move on. In a real project, check the license against your use.

7. Databases

Start with a relational database. Move away only when a requirement forces you to.

7.1 SQL vs NoSQL

7.2 The NoSQL Families

7.3 NewSQL and Distributed SQL

Systems that give SQL and transactions across many nodes, and often across regions.

Trade-off: cross-shard and cross-region transactions cost coordination latency. Keep transactions inside one shard or region where you can.

7.4 Choosing a Database

A common production shape is polyglot persistence: Postgres as the source of truth, Redis for hot reads, a search index and an analytics store fed from the primary by change data capture. Keep one system as the source of truth and treat the others as derived.

7.5 IDs

• Auto-increment: compact, ordered, but a single writer and it leaks volume.
• UUIDv4: random, no coordination, but poor for B-tree insert locality.
• UUIDv7 (RFC 9562): 48-bit millisecond timestamp plus random bits, time ordered, no coordination. PostgreSQL 18 added a native uuidv7().
• Snowflake style: 64-bit integer with timestamp, machine id and sequence. Half the size of a UUID, needs worker id management and clock care. Design in Classic Designs.

8. Indexing and Storage Engines

8.1 Indexes

An index is a separate structure that makes lookups fast at the cost of write time and space.

A composite index on (user_id, created_at) serves "this user's newest items" but not "all items by date". Every index slows writes, so add only the ones your access patterns need.

8.2 B-Tree vs LSM Tree

Rule of thumb: read-heavy with rich queries suggests B-tree, write-heavy and append-like suggests LSM.

9. Replication

Copy data to multiple nodes for availability, read scaling and locality.

Sync vs async replication:

• Synchronous: the write is acknowledged after followers have it. No data loss on leader failure, slower, and a stuck follower can block writes.
• Asynchronous: acknowledged immediately. Fast, but the last writes can be lost on failover, and reads from followers can be stale.
• Semi-synchronous: wait for one follower, a common compromise.

Replication lag anomalies and fixes:

Deep dive on quorums, failover, split brain and consensus is in Distributed Systems.

10. Sharding and Consistent Hashing

Sharding (partitioning) splits data across nodes so no node holds it all. Do it when a single primary cannot handle the data size or write rate, not before.

10.1 Strategies

10.2 Choosing a Shard Key

A good shard key has high cardinality, spreads load evenly, and matches the main query so most queries hit one shard.

Cross-shard queries and transactions are expensive. Design so the common path touches one shard, and accept scatter-gather for the rare ones.

10.3 Consistent Hashing

Place nodes on a ring by hashing. A key goes to the first node clockwise from its hash. Virtual nodes give each physical node many positions on the ring, which smooths the distribution and lets heavier machines take more.

// runnable
const crypto = require('crypto');

class HashRing {
  constructor(vnodes = 100) {
    this.vnodes = vnodes;
    this.ring = []; // sorted by hash: { hash, node }
  }
  static hash(key) {
    return crypto.createHash('md5').update(key).digest().readUInt32BE(0);
  }
  addNode(node) {
    for (let i = 0; i < this.vnodes; i++) {
      this.ring.push({ hash: HashRing.hash(`${node}#${i}`), node });
    }
    this.ring.sort((a, b) => a.hash - b.hash);
  }
  removeNode(node) {
    this.ring = this.ring.filter((e) => e.node !== node);
  }
  getNode(key) {
    if (this.ring.length === 0) return null;
    const h = HashRing.hash(key);
    let lo = 0, hi = this.ring.length;
    while (lo < hi) {
      const mid = (lo + hi) >> 1;
      if (this.ring[mid].hash < h) lo = mid + 1; else hi = mid;
    }
    return this.ring[lo % this.ring.length].node;
  }
}

const ring = new HashRing(100);
['A', 'B', 'C', 'D'].forEach((n) => ring.addNode(n));
const keys = Array.from({ length: 20000 }, (_, i) => `key-${i}`);
const before = new Map(keys.map((k) => [k, ring.getNode(k)]));
ring.addNode('E');
const moved = keys.filter((k) => before.get(k) !== ring.getNode(k)).length;
console.log('moved fraction after adding 5th node:', (moved / keys.length).toFixed(3));
// expect about 0.2 (1/5), where mod-N hashing would move about 0.8

Used by Dynamo-style stores (Cassandra, Riak), memcached client libraries, and CDN and load balancer routing. Redis Cluster uses a related idea: 16,384 fixed hash slots assigned to nodes.

10.4 Resharding and Hot Shards

• Start with more logical shards than physical nodes so you can move shards whole.
• Move data in the background with dual writes or a change log, then flip routing.
• A hot key needs a targeted fix: local cache, key splitting (key#0 to key#9), or dedicated capacity.

11. Queues and Streams

Asynchronous messaging decouples producers from consumers, absorbs spikes, and lets slow work happen off the request path.

11.1 Delivery Semantics

Design consumers to be idempotent: dedupe by message id, or make the operation naturally repeatable (set, not increment).

11.2 Key Concepts

• Partition key: decides ordering and parallelism. Messages with the same key go to the same partition and stay ordered. Choose a key with enough cardinality to spread load.
• Consumer group: partitions are divided among the consumers in a group. Max parallelism equals the partition count.
• Offset commit: commit after processing for at-least-once. Commit before for at-most-once.
• Dead-letter queue: park messages that fail repeatedly, with the error, so they do not block the queue.
• Retries: exponential backoff with jitter, separate retry topics or delay queues.
• Backpressure: bounded queues and consumer lag alerts. An unbounded queue hides overload until it explodes.
• Poison message: one bad message that fails forever, handle with DLQ.
• Compacted topics: keep only the latest value per key, useful for changelogs and state.

Recent Kafka changes worth knowing (Kafka 4.0, March 2025):

• ZooKeeper is gone. Metadata is managed by KRaft, Kafka's built-in Raft quorum, which is now the only mode.
• The new consumer group protocol (KIP-848) reduces rebalance disruption.
• Share groups (KIP-932, "Queues for Kafka") add queue-like consumption with per-message acknowledgment. At the time of the 4.0 release this was early access, so verify the status of your Kafka version before relying on it.

Trade-off: queues add latency, operational cost and eventual consistency. Do not queue work the user is waiting on unless it is slow or must be retried.

12. Object Storage

Object stores (Amazon S3, Google Cloud Storage, Azure Blob) hold large immutable blobs by key, cheaply and durably.

• Model: a flat namespace of buckets and keys. "Folders" are just key prefixes.
• Durability: designed for about 11 nines by replicating or erasure coding across zones.
• Consistency: Amazon S3 has offered strong read-after-write consistency for all operations since December 2020. Conditional writes (for example put-if-absent) were added in 2024, which enables lock-free coordination patterns on top of S3.
• Cost tiers: hot, infrequent access, archive. Use lifecycle rules to move data down.
• Large uploads: multipart upload with resumable parts, done directly from the client with pre-signed URLs so bytes never pass through your servers.
• Serving: put a CDN in front, use signed URLs for private content.

Pattern: store blobs in object storage, keep metadata (owner, key, size, checksum, permissions) in a database. Never store large blobs in the OLTP database.

13. Search

Full text search needs an inverted index: for each term, the list of documents containing it.

• Analysis: tokenize, lowercase, stem, remove stop words, at index time and query time.
• Ranking: TF-IDF and BM25 for text relevance, plus signals like recency and popularity.
• Engines: Elasticsearch, OpenSearch, Solr (all Lucene based), plus Typesense, Meilisearch. Postgres full text search suffices for small cases.
• Shards and replicas: an index is split into shards, and queries scatter to all shards and gather the top results.
• Near real time: new documents become searchable after a refresh, about one second by default in Elasticsearch.
• Sync: the search index is a derived store. Feed it from the primary DB by change data capture or an outbox, and be able to rebuild it from scratch.

Do not use a search engine as the system of record. Hybrid search that combines keywords with vector similarity is covered in AI System Design.

14. Real-Time Communication

Scaling persistent connections:

• Connections are stateful, so a gateway tier holds them, and you need a connection registry (user id to gateway id) in a fast store.
• One server can hold from tens of thousands to a few hundred thousand idle connections. Memory and file descriptors are the limits.
• Use heartbeats, reconnect with exponential backoff and jitter, and resume from the last event id.
• Fan out messages to gateways through a pub/sub layer (Redis pub/sub, Kafka, NATS).

15. APIs and Protocols

15.1 API Styles

15.2 HTTP Versions

• HTTP/1.1: one request at a time per connection, head-of-line blocking.
• HTTP/2: multiplexed streams over one TCP connection, header compression. TCP-level head-of-line blocking remains.
• HTTP/3: runs over QUIC on UDP, per-stream loss recovery, faster handshakes, connection migration for mobile networks. Widely supported by CDNs and browsers.

15.3 Pagination

Use cursor pagination for feeds and large datasets.

15.4 Idempotency

An operation is idempotent if repeating it has the same effect as doing it once. Clients retry, networks duplicate, so every write that matters needs a defense.

• GET, PUT, DELETE are idempotent by definition. POST is not.
• For POST, send an Idempotency-Key header. The server stores the key with the result and returns the stored result on a repeat.
• Flow and pitfalls are in Distributed Systems.

15.5 Other API Concerns

• Versioning: URL (/v2/), header, or additive changes only. Prefer additive and deprecate slowly.
• Rate limiting and quotas: return 429 with Retry-After. Design in Classic Designs.
• Errors: consistent shape, machine readable code, correlation id.
• Auth: OAuth 2.0 and OpenID Connect for delegated access, API keys for server to server, short-lived tokens.

16. Choosing the Right Block

17. Questions and Answers

Q1. How does a CDN decide what to cache and for how long? By cache key (URL plus selected headers) and by response headers such as Cache-Control, s-maxage and ETag. For assets, use content-hashed file names with a long max-age so invalidation is never needed.

Q2. L4 or L7 load balancer? L7 when you need to route by path or host, terminate TLS, or inspect headers. L4 for raw throughput, non-HTTP protocols, or as the first tier in front of L7 proxies.

Q3. Why is hash(key) mod N a bad sharding scheme? Changing N remaps almost every key, causing a mass data migration and cache miss storm. Consistent hashing, or a fixed number of logical shards mapped to nodes, moves only about 1/N of keys.

Q4. How do you keep cache and database consistent? You cannot make them perfectly consistent without heavy coordination. Use cache-aside with delete on write, a TTL as a safety net, and for critical keys event-driven invalidation from the database change log.

Q5. When would you pick Cassandra or DynamoDB over PostgreSQL? When the access patterns are known key lookups, the write volume or data size exceeds a single primary, and you need multi-node availability, and you can accept limited joins and transactions. Otherwise start with Postgres.

Q6. Kafka or SQS? SQS (or RabbitMQ) for simple task queues with managed operation and per-message acknowledgment. Kafka for high-throughput event streams, replay, multiple independent consumers, and stream processing.

Q7. How do you get exactly-once processing? Deliver at least once and make the consumer idempotent with a dedupe key or a natural upsert. Within Kafka, transactions plus idempotent producers give exactly-once for read-process-write between topics.

Q8. WebSocket or SSE? SSE for server to client streams (notifications, LLM tokens), it works over plain HTTP and reconnects itself. WebSocket when the client also sends frequently, such as chat and collaborative editing.

Q9. How would you serve a 5 GB video upload? Pre-signed multipart upload straight to object storage, resumable per part. An event triggers a processing pipeline, and metadata goes in a database.

Q10. What happens to your design if the cache cluster dies? Traffic falls through to the database. Protect it with circuit breakers, rate limits and load shedding, replicate the cache across zones, and keep a small in-process cache for the hottest keys.

Next: Classic Designs puts these blocks to work, and Distributed Systems goes under the hood.