Choosing a Sharding Scheme — Key Range, Fixed Shards, Hash Ranges Diagram

Each shard owns a contiguous range of sorted keys, like the volumes of a paper encyclopedia; boundaries adapt to the data
Keys are sorted inside the shard (B-tree or SSTables), so range scans and concatenated indexes work well
Pre-splitting sets boundaries on an empty database; later a shard splits or merges like the top level of a B-tree
Risk: writes to nearby keys (for example timestamps) hit one hot shard — prefix the key with a sensor ID or tenant ID

The hash need not be cryptographic: MongoDB uses MD5, Cassandra and ScyllaDB use Murmur3; language hash functions may change between processes
hash(key) % N is a trap: when N changes, most keys move
A fixed shard count (say 1,000 shards) moves whole shards between nodes and nothing else; picking the count is hard if data size varies a lot
Hash ranges split and merge on demand; consistent hashing moves as few keys as possible, in variants like rendezvous and jump hashing

Skew is an unfair split; a hot shard or hot key (a celebrity post) can leave nine nodes idle while one melts
A range-based scheme can give a hot key a shard and machine of its own
Adding random digits splits writes over 100 keys, but reads must merge all 100 — and only writes are spread
Automatic rebalancing is convenient but a false failure detection can trigger a cascading failure; a human in the loop is often safer