System Design Concepts

No fluff — visual, concise, interview-ready

🗄️ 7 · CACHING

Caching Strategies

Store frequently accessed data in a fast layer to improve latency (serve from cache → faster), throughput (reduce backend load), and cost efficiency (fewer DB calls), with trade-offs in consistency (stale data risk), invalidation (hard to update/expire), and memory usage (extra storage).

Cache-Aside (Lazy)

App Cache DB ① App reads Cache ② Cache miss! ③ App reads DB ④ DB returns data ⑤ App writes to Cache
Pro App controls logic, only requested data cached
Con Miss = 3 round trips, stale if DB updated directly

Read-Through

App Cache DB ① App reads Cache ② Cache miss ③ Cache reads DB (auto) ④ Cache stores result ⑤ Cache returns to App
Pro Simple app code — cache auto-fetches from DB
Con Cache library dependency, needs DB plugin

Write-Around

App Cache DB ① App writes DB directly ② Later: App reads Cache ③ Miss → read DB ④ Update Cache with result Cache only populated on read
Pro DB is source of truth, no stale writes in cache
Con Cache may be stale until next read miss

Write-Back (Write-Behind)

App Cache DB ① write ② async flush ⚠ Data loss if cache dies before flush Fastest writes — DB updated later in background
Pro Lowest write latency, cache+DB eventually consistent
Con Data loss risk if cache crashes before DB sync

Write-Through

App Cache DB ① write ② sync to DB ✓ Cache + DB always in sync Write latency doubles (waits for DB confirmation) 2 sync immediately ✓ Cache + DB always in sync Higher write latency (waits for DB)
Pro Reads always fresh, cache+DB in sync
Con Write latency doubles, infrequent data cached too
StrategyRead PathWrite PathConsistencyBest For
Cache-AsideApp → Cache → DB on missApp → DB (cache invalidated)EventualGeneral purpose (most common)
Read-ThroughApp → Cache (auto-fetches DB)—EventualSimpler app code
Write-AroundApp → Cache → DB on missApp → DB directlyEventualWrite-heavy, read-rarely data
Write-BackApp → CacheApp → Cache → async DBEventualHigh write throughput
Write-ThroughApp → CacheApp → Cache → sync DBStrongRead-heavy, consistency needed
Invalidation: TTL (simple, stale until expiry) · Event-driven (CDC/app triggers delete, near real-time) · Version key (new version = auto miss). Eviction: LRU (most common) · LFU · FIFO.
Thundering Herd: Cache expires → thousands hit DB simultaneously. Fix: mutex on cache miss, probabilistic early expiration, stale-while-revalidate.
Real-world: Facebook uses Memcached (TAO). Twitter caches timelines in Redis. Target: cache hit rate >95%.

Redis — Data Structures

In-memory data store — sub-ms latency, 100K–1M ops/sec. Cache + data structures + messaging

StructureCommandsUse Case
StringSET/GET/INCR/SETNX/EXPIRECounters · Session tokens · Cache · Distributed lock (SETNX) · Feature flags
HashHSET/HGET/HGETALL/HDEL/HINCRBYUser profiles · Cache objects · Shopping cart · Config store
ListLPUSH/RPOP/LRANGE/LLEN/BRPOPJob queues · Activity feeds · Recent history · Blocking queue
SetSADD/SMEMBERS/SINTER/SUNION/SISMEMBERUnique visitors · Tags · Deduplication · Common friends · Online users
Sorted SetZADD/ZRANGE/ZRANK/ZREVRANK/ZINCRBYLeaderboards · Rate limiting (sliding window) · Priority queues · Trending topics · Delayed jobs
HyperLogLogPFADD/PFCOUNT/PFMERGEApproximate unique count with 0.81% error at fixed 12KB memory · DAU · Unique page views
GeoGEOADD/GEODIST/GEORADIUS/GEOPOS/GEOSEARCHNearby drivers (Uber/Lyft) · Store locator · Location tracking · Uses Sorted Set with Geohash
StreamXADD/XREAD/XREADGROUP/XACK/XRANGEEvent log · Consumer groups · Lightweight Kafka · Audit trail

Why Redis is So Fast

Multiple design decisions compound together — 100K–1M ops/sec on a single thread

In-Memory — all data in RAM. RAM ~100ns vs disk ~10ms = 100,000x faster.
Single-Threaded Core — no locks, no context switching, no race conditions. Commands execute atomically.
Non-Blocking I/O — epoll multiplexing — single thread watches 100K+ sockets.
Efficient Internals — SDS strings, ziplist, skiplist — CPU cache-friendly.
Simple Protocol (RESP) — plain text, O(1) parsing.
Pipelining — batch multiple commands in one TCP round trip — up to 10x throughput gain.
No Query Planner — commands are direct operations — no SQL parsing, no optimizer.
RAM ~100ns · SSD ~100µs · HDD ~10ms
Single GET/SET  → 100K–1M ops/sec
With Pipelining → up to 10x gain
Bottleneck = NETWORK, not CPU → single thread is enough
Redis 6.0+: I/O threads for network read/write — still single-threaded for command execution. Redis 7.0: functions, multi-part AOF, sharded pub/sub.
Watch out: RAM-bound · avoid KEYS * / SMEMBERS on huge sets (use SCAN) · big keys block the event loop · use Redis Cluster to shard beyond single-node limits.

Redis as Cache

App checks Redis first — cache hit returns instantly, cache miss fetches from DB and populates cache

Pattern: App checks Redis first → cache hit returns instantly · cache miss → fetch from DB → write to Redis with TTL → serve.
Strategies: Cache-aside (most common — app manages cache). Write-through (write to cache + DB together). Write-back (write to cache, async flush to DB). Read-through (cache fetches from DB on miss).
Eviction Policies: allkeys-lru (evict least recently used — best for cache). allkeys-lfu (evict least frequently used). volatile-lru (only evict keys with TTL). noeviction (return error when full — for data store use).
Cache problems: Thundering herd — many requests hit DB on same cache miss → use SETNX lock or probabilistic early expiry. Cache penetration — queries for non-existent keys always miss → cache null values with short TTL. Cache avalanche — many keys expire simultaneously → add random jitter to TTLs.
Anti-patterns: No TTL — stale data forever. Cache everything — wastes RAM on cold data. No eviction policy — OOM crash. Inconsistent invalidation — cache and DB disagree.

Redis Pub/Sub & Streams

Real-time messaging built into Redis — from fire-and-forget broadcast to durable event logs

▸ Pub/Sub — Real-Time Broadcast
PUBLISH  chat:room1 "Hello!"           → sends to all current subscribers
SUBSCRIBE chat:room1                   → receives "Hello!" instantly
PSUBSCRIBE chat:*                      → pattern match — all chat channels
Subscriber joins later → NO history   → messages already gone
Guarantees: Real-time delivery (<1ms). Fan-out to all subscribers. Pattern matching.
Limitations: No persistence. No replay. No acknowledgment. No consumer groups. Fire-and-forget only.
Use cases: Figma — real-time collaboration signals. Slack — online presence indicators. Cache invalidation across app servers. Chat typing indicators.
▸ Streams — Durable Event Log (Lightweight Kafka)
XADD   orders * customer_id 5 amount 100 status "pending"  → 1704067200000-0
XGROUP CREATE orders payment-service 0                      → create consumer group
XREADGROUP GROUP payment-service consumer1 STREAMS orders > → read unprocessed
XACK   orders payment-service 1704067200000-0               → acknowledge done
XRANGE orders - +                                           → replay all entries
Guarantees: Persistence (survives restart). Replay (XRANGE from any point). Consumer groups (competing consumers). At-least-once delivery. Blocking reads (XREADGROUP BLOCK).
Limitations: Single node throughput — <100K events/sec (vs Kafka millions). RAM-bound. No cross-cluster replication. Best as lightweight Kafka when you already have Redis.
▸ Cache vs Pub/Sub vs Streams — When to Use What
FeatureCachePub/SubStreams
PurposeRead accelerationReal-time broadcastDurable event log
PersistenceTTL-basedNoneYes (AOF/RDB)
Replay❌❌✅ XRANGE
Fan-out❌✅ All subscribers✅ Consumer groups
Acknowledgment❌❌ Fire-and-forget✅ XACK
Best ForDB offload, sessionsPresence, signals, invalidationOrder pipelines, audit, IoT

Redis Persistence & High Availability

From single-node to sharded cluster — persistence, replication, and failover

▸ Persistence: RDB vs AOF

RDB (Snapshot)

Memory fork() snapshot (point-in-time) .rdb file on disk On crash: load .rdb → fast restore ⚠ Data since last snapshot is lost

AOF (Append-Only File)

Redis every write 1, 2, 3, 4, … N append-only file persisted On crash: replay AOF → near-zero data loss ✓ Hybrid (RDB + AOF) = best of both (Redis 4.0+)
▸ Redis Deployment Modes
ModeArchitectureShardingHAUse Case
Single NodeOne instanceNoNo (SPOF)Dev, small cache, non-critical
SentinelMaster + replicas + sentinel monitorsNoYes (auto-failover)HA cache, sessions, moderate load
ClusterN masters (16,384 hash slots) + replicasYesYesLarge datasets, high throughput, horizontal scale
ManagedElastiCache / MemoryDB / UpstashYesYesProduction — no ops overhead
Cluster details: 16,384 hash slots distributed across masters. Key → CRC16(key) % 16384 → slot → node. Each master has 1+ replicas. Gossip protocol for node discovery. MOVED/ASK redirects for client routing. Multi-key ops only within same slot (use hash tags: {user:123}.profile).
Redlock (distributed lock): Acquire lock on majority (N/2+1) of independent Redis nodes. Set TTL to prevent deadlock. Validate lock still held before critical section. Controversial — Martin Kleppmann argues it's unsafe (clock drift). Alternative: use etcd/ZooKeeper for strong locks.
Limitations: RAM-bound — all data must fit in memory. Single-threaded core — one slow command blocks everything. Not a primary DB — use as cache/accelerator. Async replication — data loss possible on failover (use WAIT for sync).
Real-world: Twitter — timeline cache (Redis Cluster). GitHub — job queues (Resque/Sidekiq). Snapchat — rate limiting. Pinterest — graph storage (billions of edges in Redis). Discord — presence, message cache.

CDN (Content Delivery Network)

Serve content from edge PoPs globally to improve latency (closer to users), throughput (offload origin), and availability (distributed delivery), with trade-offs in consistency (cache freshness) and invalidation (hard to purge). Pull (lazy) vs Push (proactive).

Guarantees: Low latency (<50ms from edge). DDoS absorption at edge. Origin offload. Edge computing (Cloudflare Workers) runs logic at edge.
Limitations: Dynamic/personalized content harder to cache. Cache invalidation complexity. Cost at high invalidation frequency.
▸ CDN Architecture — Edge PoPs Worldwide
ORIGIN SERVER us-east-1 PoP — Europe Frankfurt, London PoP — US West San Jose, LA PoP — Asia Singapore, Mumbai PoP — LATAM São Paulo cache fill on miss 👤👤 <20ms 👤👤 <15ms 👤👤 <30ms 👤👤 <25ms
▸ Pull CDN vs Push CDN

Pull CDN (Lazy)

Cache on first request. Cache-Control: max-age=3600
Flow: User → Edge (MISS) → Origin → Edge caches → User
Next: User → Edge (HIT, <10ms) ✓

Pro No upfront cost, auto-populates on demand
Con First request slow (cache miss), cold start
Use: General web assets, images, API responses

Push CDN (Proactive)

Pre-populate all PoPs before users request
Flow: Origin → Push to all PoPs on publish
User: User → Edge (always HIT, <5ms) ✓

Pro Zero cold starts, predictable latency
Con Storage cost, must know what to push
Use: Video segments, firmware, known-hot assets
▸ Scaling with CDN — From 1K to 1B+ Requests/Day
Tier 1: No CDN <10K req/day Browser cache only Cache-Control headers ~200ms global avg $0/mo → Tier 2: Pull CDN 10K–100M req/day Edge PoPs + Origin 90%+ cache hit ratio TTL invalidation ~20ms edge hit Cloudflare, CloudFront → Tier 3: Multi-Tier 100M–1B req/day Edge → Shield → Origin 97%+ hit, shield collapses Pub/Sub invalidation <10ms + DDoS proof Fastly, Akamai → Tier 4: Custom CDN 1B+ req/day CDN boxes inside ISPs 99%+ hit, edge compute Real-time CDC purge <5ms, zero origin Netflix Open Connect
Scaling Principles: Shield layer — intermediate cache between edge and origin that collapses duplicate misses (100 PoPs miss → 1 request to origin). Tiered TTLs — edge 60s, shield 5min, origin 1h. Request coalescing — 1000 users request same uncached asset → only 1 goes to origin. Stale-while-revalidate — serve stale, refresh async.
Pitfalls at Scale: Thundering herd — hot key expires, all PoPs hit origin. Fix: jittered TTL + coalescing. Cache stampede — popular item invalidated during spike. Fix: lock + stale-while-revalidate. Purge storms — mass invalidation overloads origin. Fix: soft purge (serve stale, refresh async).
Interview tip: Always mention cache hit ratio as the key CDN metric. A 1% improvement from 95% → 96% = 20% fewer origin requests. At Netflix scale (100B+ req/day), that's billions of saved origin calls.
Real-world: Netflix Open Connect — custom CDN in ISPs, serves 95%+ of traffic from ISP-local boxes. Cloudflare — 300+ PoPs, serves 20%+ of web traffic. CloudFront — 400+ PoPs, Lambda@Edge for compute.
Advanced Caching: Cache Warming — pre-populate cache before traffic spike (product launch, Black Friday). Multi-Level — L1 (in-process, Caffeine) → L2 (Redis) → L3 (CDN). Each level faster but smaller. CDC Invalidation — DB change → CDC event → invalidate specific cache key in real-time (no stale TTL wait). Stale-While-Revalidate — serve stale, refresh in background.
Content Delivery & Edge: CDN caches static assets at edge PoPs (Cloudflare, CloudFront). Edge Computing — run logic at edge (Cloudflare Workers, Lambda@Edge, Vercel Edge Functions). Use for: A/B testing, geo-routing, auth token validation, personalization. Reduces origin load + latency. Limitation: limited runtime, no persistent state at edge.