System design · Mechanisms: networks, replication, queues, limits, and placement

Sharding: choosing the key you will live with

Sharding splits one dataset across machines so that each holds a slice, and almost every consequence (which queries stay cheap, which become a fan-out to every shard, where the hot spots form) follows from one choice, the shard key, which is also the hardest choice in the system to reverse.

23 min read Full lesson in the app Patterns: shard-key-from-query, scatter-gather, write-sharding

What you'll be able to do

Before this: back-of-the-envelope, picking-a-datastore, replication-and-consistency

Log in to read the rest of this lesson

Sharding splits one dataset across machines so that each holds a slice, and almost every consequence (which queries stay cheap, which become a fan-out to every shard, where the hot spots form) follows from one choice, the shard key, which is also the hardest choice in the system to reverse. This walkthrough runs about 23 minutes, with runnable code you can edit and re-run as you read. A new account gets 7 days of Pro free, no card required.

Log in to continue

The first module of every track is free to read on the web: see what's open in System design.