Replication, sharding, eventual consistency, and caching

Everything in this module has avoided one box.

Stateless middles scale by photocopy. The load balancer spreads them, autoscaling breeds them, and nobody mourns a dead instance.

But the database, the stateful core and source of truth, cannot just be photocopied.

Or rather, it can. And the photocopying of truth is where distributed systems earn their reputation.

This lesson promotes three recognition-tier words from Module 7 into full concepts, pays the bill they create, and closes the caching story with one final unification.

What is replication?

ReplicationConcept · lights on your mapreplicationThe database photocopied after all: live copies on other machines, always a step behind the primary. One safety story (failover’s understudy) and one scaling story (read replicas soak up the reads while the primary keeps the writes). The catch lives in the trailing moments. means keeping live copies of the database on multiple machines.

The common arrangement has one primary, which accepts all writes, and one or more replicas, which continuously receive copies of every change.

Replication is two tools in one.

The safety story. The replica is last lesson’s understudy, the copy that failover promotes when the primary dies. That is redundancy for the one box that matters most.

The scaling story. Module 7 observed that many applications read more than they write. At scale the imbalance often turns dramatic, with ten reads for every write, sometimes far more. Point those reads at read replicas, and the primary spends itself entirely on writes. Suddenly the classic bottleneck from two lessons ago breathes.

One limit is worth stating plainly because it decides what comes next:

Replication scales reads. It does not scale writes.

Every write still lands on the one primary, and every replica must still copy it, so when the writes themselves, or the sheer size of the data, outgrow one machine, replication has no answer.

What is sharding?

The answer to that wall is shardingConcept · lights on your mapshardingWhat to do when the data itself outgrows one machine: stop copying, start splitting. Each shard takes a slice of the data and a fraction of the write load, which moves the wall replication could not. The price is wholeness, and the split is close to permanent., which splits the data itself across multiple databases.

Each shard holds only its slice, so each is small and fast again, and each absorbs only its share of the writes. The write wall moves.

The rule deciding which data lives where is called the shard key (here, the customer’s name). Real systems more often split by hashing a customer ID than by alphabet, but the alphabet makes the idea visible.

The costs are structural:

• No machine sees everything. “Total revenue today” now means asking every shard and stitching the answers together.
• Cross-shard questions get complicated. Module 7’s clean JOINs worked because everything lived in one place. A JOIN across shards is a distributed project, not a query.
• It resists undoing. Re-splitting data that is already split (because one shard grew hot, or the shard key was chosen badly) is among the most dreaded projects in engineering.

Hence the industry’s posture: teams shard as late as dignity allows. Replicas come first, a bigger primary second, and sharding when the math leaves no choice.

What is eventual consistency?

Now the bill arrives, the one you have been prepared for twice.

Eventual consistencyConcept · lights on your mapeventual consistencyThe bill for keeping live copies: a read can land on a replica the news has not reached yet, and see the past. “Eventual” promises the copies converge once the writing quiets, and promises nothing about when, though the lag is usually milliseconds. A chosen trade, not a bug, and the physics behind Module 1’s stale profile photo. is what copying truth across machines costs.

Walk the moment:

1. A write lands on the primary. The truth changes.
2. The replicas hear about it a beat later, commonly milliseconds.
3. A read that hits a replica inside that beat sees the past.

Update your profile photo, refresh, and briefly see the old one. That mystery has been open since Module 1, and Module 7 hinted at the culprit. Now you can name the physical cause. The old photo is truth still in transit between machines.

The word eventual is the promise that the copies converge. The lag is usually too short for humans to notice, until it isn’t.

And it is a choice. The alternative, making every write wait until every copy confirms, buys perfect consistency at the price of latency and availability. Now a slow replica slows every write, and an unreachable one can block them. Engineers usually judge a 200-millisecond-lagging replica a fine price for surviving failures and absorbing reads.

One slice of this trade carries a famous name you may hear, the CAP theorem: when the network itself tears and the copies cannot talk to each other, a system must choose between answering with possibly stale data and not answering at all. Recognizing the name is plenty.

Designing around the lag

Accepting eventual consistency does not mean inflicting it on every moment.

Some reads cannot tolerate the past:

Erik places an order, and the confirmation screen loads. If that read lands on a lagging replica, the screen says “no recent orders.” To the system, that is a 200-millisecond problem. To Erik, it looks like his payment vanished.

The standard surgical fix is called read-your-own-writes. Route the few reads that must see a user’s just-written data to the primary, and leave the other 99% on cheap replicas.

A useful principle is: accept the lag where it is invisible. Pay for freshness only where a human would notice.

Caching, one last time

The fourth concept needs no introduction (cachingConcept · lights on your mapcachingFinal re-encounter: keeping a copy close so the work is not done twice, told across three layers (browser, application, CDN). Every layer pays in freshness, which files caching, at last, as replication’s sibling. One family, one bargain.), but this module’s lens reveals what it always was.

A cache is a copy, trading freshness for speed.

That makes the cache and the replica siblings. The stale bestseller list and the stale profile photo are the same phenomenon, a copy that has not yet heard the news.

Line the family up:

Two of the three are copies, and every copy in every system you will ever discuss pays the same currency: copies buy speed and safety, priced in freshness. The shard pays differently, trading wholeness for room to grow.

The mental model to remember

Replication keeps live copies of the database: the understudy for failover, and read replicas that absorb read traffic. It scales reads, not writes.

Sharding splits the data itself across databases when size or write volume outgrows one machine. Each shard is fast again; no machine sees everything; undoing it is dreaded. Shard as late as dignity allows.

Eventual consistency is the bill for copying truth. Replicas lag by moments, reads can see the past, and the copies converge. It is a chosen trade, and patterns like read-your-own-writes route around the moments that cannot tolerate it.

Caching completes the family portrait. Every cache is a copy trading freshness for speed, making it replication’s sibling.

You should now be able to explain how the one box that cannot be photocopied gets copied anyway, and read any stale-data moment, anywhere, as what it is: truth in transit.