Scaling a Social Feed to 10 Million Users
The Problem
Building a social feed sounds simple. Fetch posts from people you follow, sort by time, display them. A junior engineer can build this in an afternoon.
But what happens when you have 10 million users? What happens when a celebrity with 2 million followers posts something? What happens during peak hours when everyone opens the app simultaneously?
This is where system design separates good engineers from great ones. Let's build this properly.
Understanding the Read/Write Ratio
The first question to ask in any system design interview:
"What's the read/write ratio?"
For a social feed, reads massively outnumber writes. A user might post once a day but check their feed 20-30 times. That's a 100:1 read/write ratio — and it fundamentally shapes every architectural decision we make.
The implication: Optimise aggressively for reads. We can afford to do extra work on writes if it makes reads dramatically faster.
Architecture Overview
Layer 1: Database Design
Schema
-- Users table
users (id, username, created_at)
-- Posts table
posts (id, user_id, content, created_at, media_url)
-- Follows table
follows (follower_id, followee_id, created_at)
-- Feed table (pre-computed)
feed (user_id, post_id, created_at)
The Naive Approach — Why It Fails
The obvious query for a user's feed:
SELECT p.* FROM posts p
JOIN follows f ON p.user_id = f.followee_id
WHERE f.follower_id = :user_id
ORDER BY p.created_at DESC
LIMIT 20;
This works at 1,000 users. At 10 million users, it's a full table scan joining two massive tables on every feed request. It will destroy your database.
Layer 2: The Two Approaches
Pull Model (Fan-out on Read)
When a user opens their feed, query all posts from people they follow in real time.
Pros: Simple, always fresh data, no storage overhead
Cons: Slow at scale — if you follow 500 people, that's 500 queries per feed load
Push Model (Fan-out on Write)
When a user posts, immediately push that post to all their followers' pre-computed feed caches.
Pros: Feed reads are O(1) — just read from cache
Cons: Write amplification — if a celebrity has 2M followers, one post triggers 2M cache updates
Hybrid Model — The Real Answer
For most systems at scale, the answer is hybrid:
- Regular users (< 10k followers) → Push model, pre-compute feeds
- Celebrity users (> 10k followers) → Pull model on read, merge with pre-computed feed
Layer 3: Caching Strategy
With 100:1 read/write ratio, caching is your most powerful weapon.
Redis for Feed Cache
Store each user's pre-computed feed as a Redis sorted set — scored by timestamp:
Key: feed:{user_id}
Type: Sorted Set
Score: unix_timestamp
Value: post_id
TTL: 24 hours
Feed retrieval becomes:
ZREVRANGE feed:12345 0 19 # Get 20 most recent posts
That's O(log N) — blazing fast regardless of database size.
Cache Warming
When a user hasn't been active for a while their cache might be cold. Warm it on first request:
CDN for Static Assets
Profile pictures, images, videos — serve everything from CloudFront edge locations:
Origin: S3 bucket
CDN: CloudFront
Cache-Control: max-age=31536000 (1 year for immutable assets)
Invalidation: On profile picture update
Layer 4: Database Scaling
Read Replicas
With 100x more reads than writes, add read replicas immediately:
Sharding
At 10M users, even replicated reads will eventually saturate. Shard the posts table by user_id:
Shard 0: user_ids 0 - 2,499,999
Shard 1: user_ids 2,500,000 - 4,999,999
Shard 2: user_ids 5,000,000 - 7,499,999
Shard 3: user_ids 7,500,000 - 9,999,999
Use consistent hashing so adding new shards doesn't remap all existing data.
Layer 5: Handling the Celebrity Problem
A celebrity with 2 million followers posts. In a pure push model, your feed workers need to update 2 million Redis entries — a write storm that could take minutes and spike your infrastructure.
Solution: Tiered Fan-out
Post created by @celebrity (5M followers)
↓
Check follower count threshold (e.g. 10k)
↓
Route to celebrity queue (not regular fan-out queue)
↓
Store post in celebrity_posts table
↓
On feed read: merge user's regular feed + celebrity posts
This limits write amplification while keeping reads fast.
Complete System Design
Key Decisions Summary
| Decision | Choice | Reason |
|---|---|---|
| DB type | PostgreSQL + Redis | ACID for posts, speed for feeds |
| Fan-out model | Hybrid push/pull | Balances write amplification vs read speed |
| Cache strategy | Redis sorted sets | O(1) feed reads, natural time ordering |
| Media storage | S3 + CloudFront | Infinite scale, global edge caching |
| DB scaling | Read replicas + sharding | Matches 100:1 read/write ratio |
| Celebrity posts | Tiered fan-out | Prevents write storms |
Key Takeaways
1. Always ask the read/write ratio first. It shapes every decision. 100:1 means optimise reads aggressively.
2. Pre-compute feeds for regular users. The push model trades write complexity for O(1) reads — a great trade at scale.
3. The celebrity problem is real. A naive push model breaks under high follower counts. Hybrid models solve it.
4. Cache is your best friend. Redis sorted sets are perfect for time-ordered feeds. Use them.
5. Design for the 99th percentile. Your average user isn't the problem. The celebrity with 5M followers posting during peak hours is.
The difference between a Senior and Staff Engineer here isn't knowing that you need Redis. It's knowing why you need Redis, what to store in it, and how it interacts with your consistency requirements.