Chapter 5: Caching Strategies and Cache Invalidation at the Systems Level

Why this matters

The companion Database book's Chapter 12 covered caching patterns (cache-aside, write-through, write-behind) at the level of a single service talking to a single cache. This chapter covers the systems-level questions that sit one level up: where in a multi-component system should caching live, and how do multiple layers of caching interact — questions that only arise once you're designing a whole system rather than one service's data access layer.

Where caching can live in a system, layer by layer

[BEGINNER] A well-designed system often caches at multiple layers simultaneously, each serving a different purpose:

Client (browser/app) --- Client-side cache (e.g., HTTP cache headers,
                          local storage) -- avoids a network request entirely
      |
      v
CDN --- Edge cache (companion Networking book, Chapter 8) -- avoids a
        round trip to the origin for static/slowly-changing content
      |
      v
Load Balancer
      |
      v
Service --- Application-level cache (e.g., Redis, companion Database
             book Chapter 7) -- avoids a database query
      |
      v
Database --- Database-level cache (query result caching, buffer pool --
              companion OS book Chapter 3's memory hierarchy applied to
              disk-backed data) -- avoids a disk read

[SENIOR INSIGHT] Each layer avoids a progressively more expensive operation the closer it sits to the client — this is a direct, systems-level application of the OS book's Chapter 3 memory hierarchy principle (closer, faster storage for hot data) applied across an entire distributed system's architecture rather than just within one machine. The systems-level design question is which of these layers actually earns its complexity for a specific system — caching everywhere by default adds invalidation complexity (see below) at every layer, without necessarily providing a proportionate benefit at each one.

The systems-level invalidation problem: coordinating across services

[SENIOR INSIGHT] The companion Database book's Chapter 12 covered invalidating a single cache when its underlying data changes. The systems-level version of this problem is harder: in a system with multiple services, each potentially caching the same underlying data independently, a write in one service needs to invalidate caches it may not have direct knowledge of. This is precisely why event-driven architectures (Chapter 8, and the companion Design Patterns book's Chapter 16) are so commonly paired with multi-service caching: a service that changes data publishes an event ("user 42's profile changed"), and every other service maintaining its own cache of that data subscribes and invalidates its own copy — without the writing service needing to know, or care, which other services are caching what.

Service A (writes user profile data) --publishes--> "user.profile.updated" event
                                                              |
                                    +-------------------------+-------------------------+
                                    v                                                     v
                        Service B (subscribes, invalidates          Service C (subscribes, invalidates
                        its own cached copy of user data)            its own cached copy of user data)

Read-heavy vs. write-heavy systems: fundamentally different caching value

[IN PRACTICE] Directly connecting to Chapter 3's read/write ratio elicitation: a read-heavy system (a URL shortener resolving far more often than URLs are created, a social media feed read constantly but posted to comparatively rarely) gets enormous value from caching, since a cache hit avoids re-doing expensive work that would otherwise repeat on nearly every request. A write-heavy system (an analytics ingestion pipeline, a logging system) gets comparatively little value from read-side caching, since there's relatively little repeated reading to accelerate — for these systems, design effort is better spent on write throughput (Chapter 7's database scaling, Chapter 8's async processing) than on caching strategy.

Common mistakes

1. Adding caching at every possible layer by default, without evaluating whether each specific layer's invalidation complexity is actually justified by a proportionate benefit for this specific system.
2. Designing a multi-service system's caching without an invalidation strategy that accounts for cross-service coordination, assuming each service's cache can be invalidated independently without missing the case where multiple services cache the same underlying data.
3. Applying an aggressive caching strategy to a write-heavy system, where the actual bottleneck is write throughput and caching provides comparatively little value — misallocating design effort relative to the system's actual read/write ratio.
4. Not recognizing event-driven architecture as the standard solution to cross-service cache invalidation, reinventing an ad hoc, less reliable coordination mechanism instead of reaching for the established pattern.

Checklist: designing caching at the systems level

1. Identify which layers (client, CDN, application, database) would actually benefit from caching for this specific system, rather than caching everywhere by default.
2. For any multi-service system, design cache invalidation using an event-driven approach — a writing service publishes a change event; every service with a relevant cache subscribes and invalidates independently.
3. Weight caching investment according to the system's read/write ratio (Chapter 3) — heavy investment for read-heavy systems, comparatively little for write-heavy ones.
4. Treat each caching layer's invalidation complexity as a real cost to be justified, not a free addition — this is the systems-level version of the companion Database book's caching trade-off, scaled up.

Discussion / interview questions

1. "At which layers of a system can caching live, and why might a system cache at multiple layers simultaneously?" Strong answer: Client-side, CDN/edge, application-level, and database-level — each avoids a progressively more expensive operation the closer it sits to the client, directly applying the memory hierarchy principle across a distributed system; a well-designed system caches at multiple layers when each specific layer's benefit justifies its added invalidation complexity.

2. "How would you design cache invalidation across multiple services that each independently cache the same underlying data?" Strong answer: Use an event-driven approach — the service that writes the data publishes a change event, and every other service maintaining a relevant cache subscribes and invalidates its own copy independently, without the writing service needing direct knowledge of every consumer.

3. "Why does a write-heavy system generally get less value from caching than a read-heavy one?" Strong answer: Caching accelerates repeated reads of the same data; a write-heavy system has comparatively little repeated reading to accelerate, so design effort is better spent on write throughput (database scaling, asynchronous processing) than on an aggressive caching strategy.

4. "What's the risk of adding a cache at every possible layer of a system by default?" Strong answer: Each caching layer adds real invalidation complexity; adding caching everywhere without evaluating whether each specific layer's benefit justifies that complexity produces a harder-to-reason-about system without a correspondingly larger benefit at every layer.

Chapter summary / cheat sheet

• Caching can live at multiple layers (client, CDN/edge, application, database), each avoiding a progressively more expensive operation — a systems-level application of the memory hierarchy principle.
• Cross-service cache invalidation is best handled via an event-driven approach: the writing service publishes a change event; every service with a relevant cache subscribes and invalidates independently.
• Caching investment should be weighted by a system's read/write ratio — heavy value for read-heavy systems, little for write-heavy ones.
• Every caching layer's invalidation complexity is a real cost — evaluate whether each specific layer earns its place for the system at hand, rather than caching everywhere by default.

---