Application caches, CDNs, and stateful vs. stateless

One idea has been following you since Module 4: keep a copy close, skip the trip. Today its story ends.

The idea has never changed. What has changed is the size of the trip.

Encounter one was the browser cache, saving your own device a network round-trip. Encounter two was the application cache, saving the backend a database query. Today’s third encounter scales the idea to the planet.

Then the module closes with one last distinction, stateful versus stateless, and it explains half of what you have seen since Module 8.

First, though, encounter two has some unfinished business.

What is an application cache?

An application cacheConcept · lights on your mapapplication cacheRe-encounter (Module 5 and Mini-capstone B): the backend-side store (Redis is the name to know) holding copies of expensive answers so the database is not asked the same question a thousand times a second. Fast because it lives in memory; honest about its price: speed, paid for in freshness. is a fast store where the backend keeps copies of answers it expects to need again.

You met it in Module 5 with the bestseller list, and you watched it go cold in Mini-capstone B. Now it takes its permanent place on the diagram, between the services and the database.

It still holds the same kind of thing: answers expensive to compute, requested constantly, and acceptable if slightly stale.

What is new is the box itself. An application cache is usually a separate system rather than code inside the backend, and it keeps its copies in memory, not on disk. That is Module 2’s desk rather than its filing cabinet, and it is why a cache answers so fast. The name you will hear everywhere is Redis; another is Memcached.

The working vocabulary comes in a pair:

• A cache hit: the answer was in the cache. Fast path, no database involved.
• A cache miss: it was not. The backend does the real work, then usually stores the result for next time.

Cached entries are commonly given a TTL, a time to live, after which they expire and the next request recomputes them. Choosing TTLs is choosing the trade you already know: speed, paid for in freshness.

And removing an entry early because the underlying truth changed is the cache invalidation problem again, still famously one of the hard problems at every layer where it appears.

What is a CDN?

A CDNConcept · lights on your mapCDNContent delivery network: a provider’s fleet of cache servers in hundreds of cities worldwide, holding copies of static content close to users. A user in Tokyo gets the logo from Tokyo, not Virginia. Cloudflare and Akamai are the names; nearly every site you use sits behind one., a content delivery network, is the same trick at planetary size: a provider’s fleet of cache servers, spread across hundreds of cities, each holding copies of your content close to the users near it.

Module 8 taught that distance is latency. A request from Tokyo to a server in Northern Virginia makes a round trip across the Pacific.

With a CDN, the user in Tokyo fetches the logo, the fonts, and the app’s code from a cache server in Tokyo. The trip that used to cross an ocean now crosses a neighborhood.

The mechanics are exactly the cache mechanics you just learned:

1. The first Tokyo user requests the logo. The Tokyo cache server misses, fetches it from the company’s own infrastructure (called the origin), and keeps a copy.
2. Every later Tokyo user gets a hit, served locally.
3. The copies expire on their TTLs, or are invalidated when content changes.

What belongs on a CDN? Module 4’s static-versus-dynamic split, still earning its keep. Static content, the same for everyone, caches beautifully. Erik’s cart does not.

The names to recognize are Cloudflare and Akamai. They operate famous CDNs, and the major clouds run their own. Nearly every site you use sits behind one.

Because all public traffic can flow through them, CDNs have also picked up front-door jobs: terminating TLS near the user (last lesson’s job) and absorbing certain floods of malicious traffic before they reach the origin.

When someone reports “users are seeing old data,” you can now ask a precise question. Which layer’s copy has outlived its truth, the browser’s, the application cache’s, or the CDN’s? Stale-data investigations are a walk down this exact stack.

Stateful versus stateless

The promised distinction explains several things you have been taking on faith.

Why could last lesson’s load balancer send Erik’s request to any of twelve instances? Why could autoscaling kill machines mid-afternoon without losing his cart?

The answer is the stateful versus stateless distinction.

A statelessConcept · lights on your mapstatelessA component that keeps no memory between requests; everything it needs arrives with the request or lives elsewhere. Any instance can serve anyone, which is exactly what makes it freely addable and killable, unlike a stateful one. component keeps no memory between requests. Everything it needs either arrives with the request, like the token from Module 6, or is fetched from somewhere else. Any instance can therefore serve any user, and instances can be added, removed, and replaced freely.

A statefulConcept · lights on your mapstatefulA component that holds the memory: a database, a queue, a cache. Unlike a stateless one it cannot be casually duplicated or discarded because it contains the very thing the system is meant to remember. component holds the memory: the database, the queue, the cache. These cannot be casually duplicated or thrown away because they contain the very thing the system is supposed to remember.

Every modern diagram follows the same deep pattern:

A disposable, duplicable, stateless middle pairs with a small, carefully guarded, stateful core.

The backend instances hold nothing, so they scale like Module 8’s cattle. The state lives in a few purpose-built systems that are protected, backed up, and scaled with far more care.

Module 1 asked: what does the system remember? Architecture’s mature answer is as little as possible, in as few places as possible.

Is the cache stateful?

One nuance is left before the whiteboard gets assembled.

The cache holds state too. So why did Mini-capstone B treat the noon cache restart as survivable, when losing the database would be a catastrophe?

The answer is that not all state is equally precious.

• The database holds authoritative state: the only copy of the truth. Losing it means losing orders, accounts, and money.
• The cache holds reconstructible state: copies of answers that can be recomputed from the truth. Losing it means slowness (a storm of cache misses), not loss.

A useful principle is: ask not just whether a component holds state, but whether its state could be rebuilt.

That question (what survives when this box restarts?) separates readers of diagrams from people who merely look at them.

The whiteboard, complete

Module 9’s pieces now assemble into one readable whole:

Every box on that diagram is now a word you own, with a job you can explain and a failure you can predict.

The mental model to remember

An application cache keeps copies of expensive answers near the backend. Hits are fast. Misses do the real work and store it. TTLs and invalidation keep the copies honest.

A CDN is the same idea at planetary scale: cache servers in hundreds of cities serving static content from near the user, filled from the origin on first request.

The caching story is one bargain at three sizes: speed, purchased with freshness. And stale-data bugs are a walk down the three layers.

A stateless component remembers nothing between requests and can be freely duplicated and replaced. A stateful component holds the memory and must be guarded.

The modern pattern: a stateless, disposable middle around a small stateful core, and within the core, authoritative state (the database) is treated differently from reconstructible state (the cache).

You should now be able to read a full architecture diagram end to end (doors, middles, cores, queues, topics, and caches) and answer the deciding question for any box: what does it remember, and what happens when it restarts?