Sharding Secondary Indexes — Local Versus Global Diagram

Local (Document-Partitioned) Indexes

Each shard indexes only its own records, so a write touches exactly one shard
A lookup without the partition key must query every shard and merge the answers
Scatter queries amplify tail latency, and adding shards grows storage but not query throughput
Used by MongoDB, Riak, Cassandra, Elasticsearch, SolrCloud, and VoltDB

Global (Term-Partitioned) Indexes

The index is itself sharded, by the indexed value: terms a–r on one shard, s–z on another
A single-term query reads the postings list from one shard; fetching the records still touches many shards
Combining terms means intersecting postings lists over the network — slow when the lists are long
CockroachDB, TiDB, and YugabyteDB use global indexes; DynamoDB updates them asynchronously, so reads can be stale

Pitfalls

A global index write may touch several index shards: one write becomes many, and consistency needs distributed transactions
Hand-rolled indexes in application code drift out of sync after races and partial write failures
Choosing local versus global is a trade-off between cheap writes and cheap reads