Core Building Blocks

Master the building blocks of system design — load balancers, caches, CDNs, message queues, API gateways, and the components you'll use in every system design interview.

Premium Course Content

This lesson is part of a premium course. Upgrade to Pro to unlock all premium courses and content.

Access all premium courses
1000+ AI skill templates included
New content added weekly

← Back to course overview

🔄 Quick Recall: In the previous lesson, you learned estimation — the math that determines your system’s scale. Now you’ll learn the building blocks — the components you’ll use in every system design to handle that scale.

Building blocks are the reusable components that appear in almost every system design. Understanding when and why to use each one — not just what they do — is what separates strong candidates from candidates who just draw boxes.

Building Blocks Overview

Block	What It Does	When To Use	Key Trade-off
Load Balancer	Distributes traffic across servers	Multiple servers handling the same request type	Adds latency hop; single point of failure unless redundant
Cache	Stores frequently accessed data in memory	Read-heavy workloads (>10:1 read/write ratio)	Stale data; cache invalidation complexity
CDN	Serves static content from edge locations	Static assets (images, CSS, JS) or cacheable API responses	Cache invalidation delay; cost per request
Message Queue	Decouples producers from consumers	Async processing, different processing speeds	Eventual consistency; added infrastructure
API Gateway	Single entry point for clients	Multiple microservices; authentication, rate limiting	Added latency; single point of failure
Rate Limiter	Controls request rate per user/IP	Preventing abuse, protecting downstream services	Legitimate users may be throttled during traffic spikes

Caching Deep Dive

AI prompt for caching design:

Design the caching strategy for [SYSTEM]. Read/write ratio: [X:1]. Data size: [Y]. Latency requirements: [Z]. Help me decide: (1) What to cache (which data, what percentage), (2) Cache pattern — cache-aside, write-through, write-behind, (3) Cache size and TTL, (4) Invalidation strategy — TTL expiry, event-driven invalidation, or both, (5) What happens on cache miss (thundering herd protection), (6) What happens when the cache goes down (graceful degradation).

Caching patterns:

Pattern	How It Works	Best For
Cache-Aside	App reads cache first; on miss, reads DB and populates cache	General purpose, read-heavy
Write-Through	App writes to cache AND DB simultaneously	Data that’s read immediately after writing
Write-Behind	App writes to cache; cache writes to DB asynchronously	Write-heavy, can tolerate brief inconsistency
Read-Through	Cache automatically loads from DB on miss	When cache manages its own data loading

Message Queues

AI prompt for queue design:

Design the message queue architecture for [SYSTEM]. Use cases: [LIST — e.g., “order processing, email notifications, analytics events”]. For each use case: (1) Which queue pattern — point-to-point or pub/sub, (2) Delivery guarantee — at-most-once, at-least-once, or exactly-once, (3) Ordering requirements — does message order matter? (4) Dead letter queue — what happens to messages that fail processing, (5) Consumer scaling — how many consumers, how to avoid duplicate processing.

✅ Quick Check: Your system uses a message queue for order processing. The queue has 3 consumers. A message for Order #123 is picked up by Consumer A, which crashes before completing processing. What happens to Order #123? (Answer: It depends on the acknowledgment model. With auto-ack, the message is lost (at-most-once). With manual-ack, the message returns to the queue after a timeout and is picked up by Consumer B (at-least-once). For order processing, you need at-least-once + idempotent processing to ensure every order is processed exactly once even if a consumer crashes.)

CDN and Static Content

AI prompt for CDN strategy:

Design the CDN strategy for [SYSTEM]. Content types: [LIST — images, videos, CSS/JS, API responses]. Generate: (1) What to serve from CDN (static vs. dynamic), (2) Cache TTL per content type, (3) Cache invalidation strategy (purge, versioned URLs, stale-while-revalidate), (4) Origin shield configuration, (5) Estimated bandwidth savings.

Key Takeaways

Cache strategically based on access patterns: 20% of data serves 80% of reads (Pareto principle) — cache that 20% and one Redis instance can replace 9 database replicas, reducing cost by 90%
Load balancing algorithm matters when requests have different costs: round-robin assumes equal requests, least-connections adapts to actual server load — always explain why you chose a specific algorithm
Message queues decouple synchronous user-facing operations from slow asynchronous side effects: the user gets a fast response (50ms) while email, analytics, and notifications process asynchronously
For every building block, state the trade-off: caching trades consistency for speed, message queues trade immediacy for reliability, CDNs trade freshness for latency. Interviewers want to hear what you’re giving up
AI explains each building block at your level and helps you practice justifying component selection: “I added a cache because our 100:1 read/write ratio and 50K QPS makes database reads the bottleneck”

Up Next

In the next lesson, you’ll learn data storage — choosing between SQL and NoSQL, designing schemas, sharding strategies, and the database decisions that determine your system’s scalability.

Knowledge Check

1. Your system handles 50,000 reads per second but only 500 writes per second. The database handles 5,000 reads/second. Without a cache, you need 10 database replicas just for reads. What's the caching strategy?

Cache everything — put a Redis layer in front of the database and cache all reads Cache strategically based on access patterns. Analysis: (1) Identify the hot data — in most systems, 20% of data serves 80% of reads (Pareto principle). Cache that 20%. (2) Choose the caching pattern based on read/write ratio: with 100:1 read/write, use Cache-Aside (lazy loading): read from cache first, on miss read from DB and populate cache. Writes go to DB and invalidate cache. (3) Size the cache: if 20% of data is 10GB, that fits in a single Redis instance (Redis handles 100K QPS). (4) Set TTL based on data freshness requirements: 60 seconds if users can see slightly stale data, no TTL if consistency is critical (use cache invalidation on writes instead). Result: one Redis instance replaces 9 database replicas, reducing cost by 90% while serving reads in 1ms instead of 50ms Add read replicas — scale the database horizontally instead of adding a cache

2. You add a load balancer to distribute traffic across 4 API servers. The load balancer uses round-robin. One server is handling file uploads (slow, CPU-intensive) while the others handle quick API calls. Users of the slow server complain about latency. What's wrong with round-robin?

Use random distribution instead — it avoids the sequential pattern Round-robin distributes requests equally by COUNT but not by LOAD. A file upload takes 10 seconds of server time, while a quick API call takes 50ms — round-robin doesn't know this. Better strategies: (1) Least connections — route to the server with the fewest active connections. The server processing a 10-second upload accumulates connections; new requests go elsewhere. (2) Weighted round-robin — assign weights based on server capability or current load. (3) Separate the workloads — route file uploads to dedicated servers and API calls to separate servers. AI explains each strategy: 'For a mixed workload with file uploads (slow) and API calls (fast), least-connections is the best default because it naturally balances actual server utilization, not just request count' Add more servers — with 8 servers, the impact of one slow request is halved

3. Your system sends an email notification whenever a user places an order. The email service takes 2-5 seconds to respond. Without optimization, every order API call takes 2-5 extra seconds. What's the design pattern?

Send the email in a background thread — the API call returns immediately Use a message queue to decouple the order service from the email service. (1) User places order → Order Service writes to database AND publishes a message to the queue (10ms total) → returns success immediately. (2) Email Service reads from the queue → sends the email (2-5 seconds, but the user isn't waiting). Benefits: (1) Order API response time drops from 2-5 seconds to 50ms — the user isn't waiting for email delivery. (2) If the email service is down, messages queue up and are processed when it recovers — no orders are lost. (3) The email service can process at its own pace — during a flash sale with 10,000 orders/minute, emails queue and send gradually instead of overwhelming the email server. The trade-off: eventual delivery (email arrives seconds after order, not simultaneously) and added infrastructure (the queue itself). In this case, the trade-off is clearly worthwhile Cache the email service response — if it's the same template, the response time improves

Answer all questions to check