13 — Where to Cache: From the Client to the Database

"Cache is cache" was my caching model, which made the five possible locations feel interchangeable. Writing them down separated the layers cleanly: caching happens at every layer from the browser to the database, and each layer caches a different thing for a different reason. [1][2] The strategy notes (cache-aside, write-through, etc.) describe _how_ a cache is populated; these notes describe _where_ the cache sits and what it protects.

The framing that landed is that each layer in the path between user and database is a candidate cache, and the cache at each layer answers a distinct question. The browser cache asks "have I fetched this page before?" The CDN cache asks "is there a copy near the user?" The web-server cache asks "can I return this without bothering the app?" The database cache asks "have I already computed this query?" The application cache asks "is this hot data already in memory?" A fast system usually has caches at several of these layers, each doing a different job.

The five layers, in request order

A request travels from the user's device to the database and back. At each hop, a cache can short-circuit the rest of the journey [1][2]:

• Client caching stores frequently-accessed data on the client's device rather than the server [3]. The classic example is the browser caching pages and assets — when a user revisits, the browser returns the local copy instead of re-fetching. Mobile apps can cache data on-device too. The win: no network at all, the fastest possible hit. The cost: the potential for stale data if the client cache is not managed, plus memory/disk use on the user's device.
• CDN caching is the globally-distributed edge cache covered in the DNS-and-CDN notes. When a user requests content, the CDN checks a nearby edge server first; on a hit, it serves from there. On a miss, it fetches from the origin and caches the result. The win: reduced distance, reduced origin load.
• Web server caching sits at the reverse-proxy layer. Reverse proxies and caches like Varnish can serve static _and_ dynamic content directly, returning a response without ever contacting the application server [4]. The win: the application never sees the request at all.
• Database caching stores frequently-accessed query results so the database does less work. Instead of re-running an expensive query, the cached result is returned [5]. Most databases also have internal buffer caches (in-memory pages) that are transparent to the application.
• Application caching is the in-memory key-value layer (Memcached, Redis) between the application and the database. Because data is held in RAM, it is much faster than a disk-backed database [6]. The cost is that RAM is limited, so eviction algorithms like least-recently-used (LRU) keep hot data in memory and evict cold entries [6]. Redis adds persistence options and built-in data structures (sorted sets, lists) on top of the basic key-value model.

What each layer is for

The reason this matters is that the layers are not redundant — each protects a different bottleneck. The client cache saves the network round trip. The CDN cache saves the trip to the origin. The web-server cache saves the application from being invoked. The application cache saves the database query. The database cache saves the query computation. A request that hits the client cache never leaves the device; a request that misses everything and hits the database pays every layer's cost.

The corollary I had to learn the hard way: caching at the wrong layer does not help. Caching a database query result in the application cache does nothing for users on the other side of the world if the latency floor is the trip to the origin — that needs a CDN. Caching at the CDN does nothing for a per-user personalized response that cannot be shared — that needs an application cache keyed by user. The cache has to sit at the layer whose bottleneck is actually the problem.

How I use this

When something is slow, I name the layer where the latency is being paid before adding any cache. If the floor is geographic distance, the fix is the CDN (or client cache). If the app server is the bottleneck, the fix is the web-server or application cache. If the database is the bottleneck, the fix is the application or database cache. And I avoid file-based caching — the roadmap's parting advice, because file caches make cloning and auto-scaling harder [6]. The discipline of matching the cache layer to the actual bottleneck is what turns caching from a guess into a tool.

References

[1] MDN, "HTTP caching," Mozilla Developer Network, 2024. [Online]. Available: https://developer.mozilla.org/en-US/docs/Web/HTTP/Caching

[2] Amazon Web Services, "Database caching," AWS. [Online]. Available: https://aws.amazon.com/caching/database-caching/

[3] Prisma, "Introduction to database caching," Prisma Data Guide. [Online]. Available: https://www.prisma.io/dataguide/managing-databases/introduction-database-caching

[4] D. Martin, "Reverse proxy (web server)," system-design-primer (open source), 2024. [Online]. Available: https://github.com/donnemartin/system-design-primer#reverse-proxy-web-server

[5] "Database caching strategies," Medium, 2022. [Online]. Available: https://medium.com/@sesmiat/database-caching-strategies-f5e40c3c9b74

[6] D. Martin, "Application caching," system-design-primer (open source), 2024. [Online]. Available: https://github.com/donnemartin/system-design-primer#application-caching