System Design Case Study

How does Twitter deliver posts to 400M+ users' timelines in real-time?

?? Design a timeline delivery system: 400M users, celebrity fan-out, <5s delivery
Concepts Involved
Redis Kafka Pub/Sub Sharding Caching Consistent Hashing

Problem Statement

How does a social platform deliver posts to 400M+ users' timelines in real-time, handling the asymmetry between users with few followers and celebrities with millions, while keeping timeline delivery under 5 seconds?

Core challenge: A celebrity with 100M followers posts a tweet. You cannot write to 100M timelines simultaneously · that's a fan-out explosion. But followers expect to see it within seconds. How do you balance write amplification vs read latency?
400M+
active users
timelines to serve
100M
max followers
celebrity problem
<5s
delivery target
post ? visible in feed
500K+
tweets / sec peak
during major events

Functional Requirements

Must Have

1. User posts a tweet ? appears in all followers' timelines
2. Timeline is ranked (not purely chronological) · relevance + recency
3. Real-time updates · new tweets appear without manual refresh
4. Handle celebrity asymmetry · users with 100M followers
5. Support infinite scroll with pagination
6. Engagement signals update in real-time (likes, retweets, replies count)

Out of Scope

? Tweet composition and media upload
? Search and trending topics
? Direct messages
? Notifications
? Content moderation

Functional Requirements

Must Have

1. User posts a tweet ? appears in all followers' timelines
2. Timeline is ranked (not purely chronological) · relevance + recency
3. Real-time updates · new tweets appear without manual refresh
4. Handle celebrity asymmetry · users with 100M followers
5. Support infinite scroll with pagination
6. Engagement signals update in real-time (likes, retweets, replies count)

Out of Scope

? Tweet composition and media upload
? Search and trending topics
? Direct messages
? Notifications
? Content moderation

Non-Functional Requirements

PropertyTargetDesign Impact
Latency<5s tweet ? visible in follower timelineFan-out workers must process within seconds, not minutes
Read Latency<100ms timeline loadPre-computed timeline in Redis (no DB query on read)
Throughput500K tweets/sec peak (major events)Kafka + parallel fan-out workers, auto-scale on lag
Availability99.99%Redis replicas, multi-AZ, graceful degradation (serve stale)
ConsistencyEventual · tweets may appear 1-5s lateFan-out is async. Acceptable: not a banking system.
Scalability400M users, linear scale with user countShard timeline cache by user_id. Add Redis nodes as users grow.

High-Level Architecture

Hybrid fan-out: write for normal users, read for celebrities

Twitter Timeline Fan-out · End-to-End Architecture WRITE LAYER · Tweet Ingestion & Fan-out Decision Tweet API 500K tweets/sec peak persist to Tweet DB Kafka tweet-events topic durable, replayable Fan-out Decision followers < 10K ? WRITE path (99%) followers > 10K ? READ path (1%) Social Graph (FlockDB) follower lookups per tweet sharded by user_id | avg 200 followers Hybrid: 99% of tweets fan-out on write (fast reads) | 1% celebrities merge on read (no write explosion) FAN-OUT LAYER · Write Path & Celebrity Storage Fan-out Workers (~990 instances) For each follower of tweet author: ZADD timeline:{userId} {timestamp} {tweetId} 99M Redis writes/sec peak | parallel across shards ZREMRANGEBYRANK 0 -801 (trim to 800 entries per user) Redis Cluster ~2.5 TB across 100+ shards Sorted Set per user (score=ts) 400M user timelines cached cache hit rate: ~99% Celebrity Path (Read) Store tweet in DB only No fan-out (100M writes avoided) Merged at read time per user query: "my celebrity follows" READ LAYER · Timeline Assembly & Ranking Timeline Mixer ? ZREVRANGE timeline:{userId} 0 19 (pre-computed) ? Query celebrity tweets user follows ? Merge + deduplicate + sort by score ML Ranking Service Personalized relevance scoring Features: engagement, recency, affinity TensorFlow Serving | <50ms p99 Client App Ranked timeline feed Infinite scroll pagination WebSocket for real-time Real-time SSE / WebSocket new tweets push no manual refresh Hybrid Fan-out Strategy · By the Numbers Fan-out on WRITE (followers < 10K) 1 tweet ? ZADD to each follower's Redis sorted set ? Read latency: <5ms (O(1) from cache) ? Write amplification: 1 tweet ? 200 avg writes Covers 99% of users (avg ~200 followers) Peak: 500K · 200 · 0.99 = 99M writes/sec Fan-out on READ (followers > 10K) Store tweet once in DB, merge at read time ? No write amplification for celebrities ? Read adds ~20ms (query + merge + rank) Only top ~1% of users (celebrities, brands) Avoids: 100M follower · 1 tweet = 100M writes
Why hybrid? Pure fan-out-on-write: a celebrity tweet triggers 100M Redis writes = minutes of lag. Pure fan-out-on-read: every timeline load queries all followed users = slow reads. Hybrid: 99% of tweets fan-out on write (fast reads), celebrities merge on read (no write explosion).
Timeline cache: Redis sorted set per user. Score = timestamp. ZREVRANGE for latest N tweets. Trim to 800 entries (older tweets fetched from DB). Cache hit rate: ~99% for active users.
Failure modes: Fan-out worker lag ? tweets delayed for some followers (acceptable: eventual). Redis node failure ? rebuild from DB (cold start ~30s). Celebrity tweet during event ? read path handles gracefully (no write storm).
Real-world: Twitter uses this exact hybrid approach. Threshold is ~10K followers. Instagram uses similar fan-out-on-write for feed. Facebook uses primarily fan-out-on-read with aggressive caching (TAO).

Scale Estimation

Back-of-envelope math for timeline delivery

Given: 400M active users · 500K tweets/sec peak · avg 200 followers · top 1% have >10K followers
StepDerivationResultDesign Impact
1Fan-out writes (normal): 500K tweets · 200 followers · 99%~99M Redis writes/secRedis Cluster with 100+ shards for timeline cache
2Celebrity tweets skipped: 500K · 1%~5K tweets/sec (read path)Merged at read time · no write amplification
3Timeline reads: 400M users · 10 opens/day · 86400~46K timeline reads/secRedis cache hit rate ~99% · most reads served from cache
4Timeline cache size: 400M users · 800 tweet_ids · 8 bytes~2.5 TB RedisSorted sets, trim to 800 entries per user
5Fan-out worker count: 99M writes/sec · 100K writes/worker~990 fan-out workersStateless, horizontally scalable, Kafka-driven
6Tweet storage: 500K/sec · 1KB · 86400~43 TB/daySharded by tweet_id, replicated 3·

Data Model

Timeline cache (Redis) + Tweet store (MySQL/Manhattan) + Social graph (who follows whom)

Redis Sorted Set: timeline:{user_id} score = timestamp | member = tweet_id | ZREVRANGE 0 19 ? latest 20 tweets tweet_901 score: 1704067200 tweet_899 score: 1704067180 tweet_895 score: 1704067100 ... tweet_102 score: 1704000000 ? oldest (trimmed at 800) ZADD timeline:user123 1704067200 tweet_901 Write: ZADD (fan-out) Read: ZREVRANGE 0 19 Trim: ZREMRANGEBYRANK 0 -801 Cache hit: 99% Memory: 400M users · 800 tweet_ids · 8 bytes = ~2.5 TB Redis (sharded across 100+ nodes) 
// --- Timeline Cache (Redis Sorted Set) ---
Key:   timeline:{user_id}
Value: Sorted Set · score=timestamp, member=tweet_id
Ops:   ZADD timeline:user123 1704067200 tweet_abc    // fan-out write
       ZREVRANGE timeline:user123 0 19               // get latest 20
       ZREMRANGEBYRANK timeline:user123 0 -801       // trim to 800

// --- Tweet Store (Sharded MySQL / Manhattan) ---
tweets {
  tweet_id:    BIGINT (Snowflake ID)    -- partition key
  user_id:     BIGINT
  text:        VARCHAR(280)
  media_urls:  JSON
  created_at:  TIMESTAMP
  reply_to:    BIGINT NULL
  retweet_of:  BIGINT NULL
  like_count:  INT (eventually consistent counter)
  rt_count:    INT
}

// --- Social Graph (who follows whom) ---
follows {
  follower_id:  BIGINT
  followee_id:  BIGINT
  created_at:   TIMESTAMP
}
-- Index: (followee_id) ? get all followers (for fan-out)
-- Index: (follower_id) ? get all following (for timeline merge)

// --- Celebrity List (cached) ---
celebrities: SET of user_ids with followers > 10K
-- Updated hourly, cached in memory on fan-out workers

Resilience & Edge Cases

FailureImpactRecovery
Fan-out worker lagTweets delayed for some followersAcceptable: eventual delivery. Auto-scale workers on Kafka lag.
Redis node failureTimeline cache lost for shardRebuild from DB: query follows ? fetch recent tweets ? populate cache (~30s cold start)
Celebrity tweets during eventCould overwhelm read pathPre-compute celebrity timelines, cache aggressively, CDN for trending tweets
New follower of celebrityMissing old tweets in timelineOn follow: backfill last N tweets from celebrity into follower's timeline
User unfollowsStale tweets in timelineLazy removal: filter at read time. Async cleanup in background.

Tech Stack & Tradeoffs

ComponentTechnologyWhyRejected
Timeline CacheRedis Cluster (sorted sets)Sub-ms reads, atomic ZADD, TTL, shardingMemcached (no sorted sets), DynamoDB (higher latency)
Tweet StoreManhattan (Twitter's KV) / MySQLSharded by tweet_id, high write throughputCassandra (less familiar), PostgreSQL (single-node limits)
Fan-out QueueKafkaDurable, replayable, handles 500K tweets/secRabbitMQ (no replay), SQS (no ordering)
Social GraphFlockDB (Twitter's graph DB)Optimized for follower lookups, shardedNeo4j (doesn't scale), MySQL (slow for graph traversal)
RankingML model (TensorFlow Serving)Personalized relevance scoring per userChronological only (lower engagement)
Real-time PushWebSocket / Server-Sent EventsNew tweets appear without refreshPolling (wasteful, laggy)

Interview Cheat Sheet

The 8 things to say for timeline/feed design

1. Hybrid fan-out · write for normal users (<10K followers), read for celebrities (>10K)
2. Redis sorted set per user · score=timestamp, ZREVRANGE for latest, trim to 800
3. Fan-out workers consume from Kafka · tweet event ? lookup followers ? ZADD to each timeline
4. Celebrity merge at read time · query "my celebrity follows" ? fetch their recent tweets ? merge + rank
5. ML ranking · not purely chronological. Score by relevance, engagement prediction, recency
6. Cache hit rate ~99% · active users' timelines always warm. Cold start on first login after long absence.
7. Social graph sharded by user_id · follower lookup is the hot path during fan-out
8. Engagement counters eventually consistent · likes/RTs updated async, not blocking delivery