Sharding
Horizontal scaling, shard keys, and cluster architecture
Skills taught: mongodb.sharding
What is Sharding?
Sharding is MongoDB's approach to horizontal scaling. It distributes data across multiple machines called shards. Unlike replication (which keeps copies of the same data on multiple nodes), sharding divides the data so each shard holds a different subset.
Sharded Cluster Architecture:
Client Applications
│
▼
┌─────────────┐
│ mongos │ ← Query Router (stateless)
│ (router) │
└─────────────┘
│
┌────┴────┐
▼ ▼
┌──────┐ ┌──────┐ ← Shards (each is a Replica Set!)
│Shard1│ │Shard2│
│ RS │ │ RS │
└──────┘ └──────┘
┌────────────────────┐
│ Config Servers │ ← Metadata (which chunks on which shard)
│ (Replica Set) │
└────────────────────┘
Shard 1 holds: users where _id hashed to range A-M
Shard 2 holds: users where _id hashed to range N-Z
The Shard Key
The shard key is the field (or fields) MongoDB uses to distribute documents across shards. Choose it carefully — the shard key cannot be changed after sharding. A good shard key has high cardinality, even distribution, and matches your query patterns.
// In mongos shell: enable sharding on a database
sh.enableSharding('mydb');
// Shard a collection by a HASHED key (best for even distribution)
sh.shardCollection('mydb.events', { userId: 'hashed' });
// Hashed sharding: MongoDB hashes userId, distributes evenly
// All writes go to random shards — no hotspots
// Downside: range queries on userId go to ALL shards
// Shard a collection by RANGE key (use when range queries are common)
sh.shardCollection('mydb.orders', { customerId: 1 });
// Range sharding: orders with nearby customerId go to same shard
// Good: orders.find({ customerId: { $gte: 'A', $lte: 'M' } }) → 1 shard
// Bad risk: if most writes have customerId starting with 'A', creates a hotspot
// Compound shard key (more cardinality, avoids hotspots)
sh.shardCollection('mydb.logs', { region: 1, timestamp: 1 });
// First field: 'region' scopes the shard. Second field: 'timestamp' distributes within region
Targeted vs Scatter-Gather Queries
Including the shard key in your query filter allows MongoDB to route the query to a single shard (targeted query). Queries without the shard key must be sent to all shards and results merged (scatter-gather), which is much slower.
// Assume collection is sharded by { userId: 'hashed' }
// TARGETED — includes shard key (userId)
// mongos routes to exactly ONE shard
const userOrders = await orders.find({ userId: 'user123' }).toArray();
// Fast: one network hop to one shard
// SCATTER-GATHER — no shard key in filter
// mongos sends to ALL shards, merges results
const pendingOrders = await orders.find({ status: 'pending' }).toArray();
// Slow: N network hops to N shards, results merged on mongos
// Gets worse as you add more shards!
// Best practice: always include the shard key in your most frequent queries
// Design shard key to match your highest-frequency query pattern
// Check if a query is targeted:
const plan = await orders.find({ userId: 'user123' }).explain();
console.log(plan.queryPlanner.winningPlan.shards.length); // 1 = targeted, N = scatter-gather
- Hotspot
- A shard receiving disproportionately more writes or reads than other shards, defeating the purpose of sharding. Caused by monotonically increasing shard keys (like timestamps or auto-increment IDs) which always route new writes to the same shard.
Avoiding Hotspots
// BAD shard key: timestamp (monotonically increasing)
// All new events go to the shard holding the latest range
// → one shard is always the 'hot' shard receiving all new writes
sh.shardCollection('mydb.events', { timestamp: 1 });
// BAD shard key: _id with ObjectId
// ObjectId's first 4 bytes are a timestamp — same hotspot problem!
sh.shardCollection('mydb.events', { _id: 1 });
// GOOD: hashed _id — distributes writes evenly across all shards
sh.shardCollection('mydb.events', { _id: 'hashed' });
// MongoDB hashes the ObjectId, distributing writes randomly
// GOOD: compound with high-cardinality field first
// region is low-cardinality (8 regions) → use as zone partition
// userId is high-cardinality → distributes within each region
sh.shardCollection('mydb.events', { region: 1, userId: 'hashed' });
Hashed vs Range Sharding
- Hashed Sharding: MongoDB hashes the shard key and distributes by hash value
- Range Sharding: Documents with adjacent shard key values stored on same shard
Warning: Never shard before you need to. Sharding adds complexity: transactions span more infrastructure, $lookup across shards is restricted, debugging is harder. Try vertical scaling (larger instance) and query/index optimization first. Most applications never need sharding.
Frequently Asked Questions
When should I shard?
Only shard when a single replica set cannot handle your data volume or write throughput. First optimize queries and indexes. Then try a larger instance (vertical scaling). Only when neither works, consider sharding. Most applications — even at significant scale — never need it.
Can I change the shard key after sharding?
Starting MongoDB 4.4, you can reshard a collection with a new shard key, but it's an expensive operation. In earlier versions, the shard key was permanently immutable. Even now, choose your shard key carefully upfront to avoid a costly reshard operation.
Does MongoDB Atlas handle sharding automatically?
Yes. With Atlas, you can enable sharding on a cluster and Atlas handles the infrastructure. You still choose the shard key for your collections. Atlas also provides Global Clusters for geographic data distribution (e.g., EU user data stays on EU shards).