Chapter 12: Caching Layers and Invalidation Strategies

Why this matters

A cache is the single highest-leverage tool for reducing database load and improving read latency — and also, famously, one of "the two hard problems in computer science" (cache invalidation, alongside naming things). This chapter covers caching patterns concretely, using Redis (Chapter 7) as the caching layer, and treats invalidation as the actual hard, central problem it is, not an afterthought.

Cache-aside (lazy loading): the most common pattern

[BEGINNER] In cache-aside, the application checks the cache first; on a miss, it queries the database, then populates the cache for next time:

def get_product(product_id):
    cached = redis_client.get(f"product:{product_id}")
    if cached is not None:
        return json.loads(cached)  # cache HIT -- no database query at all

    # cache MISS -- fall through to the database
    product = db.query("SELECT * FROM products WHERE id = %s", [product_id])
    redis_client.setex(f"product:{product_id}", 3600, json.dumps(product))  # cache for 1 hour
    return product

Trade-off: simple, and the cache only ever holds data that's actually been requested (no wasted caching of unused data) — but the first request for any given key always pays the full, uncached database query cost (a "cold" cache miss), and a popular key's cache expiring under high concurrent load is exactly the thundering herd problem from the companion OOP and Networking books (many concurrent requests all discover the miss simultaneously and all query the database at once) — the single-flight/locking pattern from those books applies directly here as the standard mitigation.

Write-through: keeping the cache always current

[BEGINNER] In write-through caching, every write goes to the cache and the database together (typically the cache first, or both atomically), so the cache is never stale relative to the database:

def update_product_price(product_id, new_price_cents):
    db.execute("UPDATE products SET price_cents = %s WHERE id = %s", [new_price_cents, product_id])
    redis_client.setex(f"product:{product_id}", 3600, json.dumps({"id": product_id, "price_cents": new_price_cents}))
    # Cache is updated in the SAME operation as the database write -- never stale.

Trade-off: eliminates the staleness window cache-aside can have between a write and the next cache refresh, but every write now pays the latency cost of updating both the database and the cache, and data that's written but never subsequently read still consumes cache space unnecessarily.

Write-behind (write-back): fastest writes, real risk

[SENIOR INSIGHT] In write-behind caching, a write goes to the cache immediately (fast, low-latency acknowledgment to the client) and is asynchronously flushed to the database afterward, in the background — trading durability for write speed, similar in spirit to the companion Operating System book's asynchronous replication trade-off (Chapter 10 of this book): a crash between the cache write and the eventual database flush means that write is lost entirely, never having reached durable storage. This pattern is used deliberately in specific, narrow scenarios (very high write throughput where some data loss risk on rare crashes is an accepted, explicit trade-off) — it should never be a default choice, precisely because of this real, absolute data-loss risk.

Invalidation strategies: the actual hard problem

[BEGINNER] Every caching pattern above needs a strategy for handling data that becomes stale — this is the genuinely hard part of caching, and the reason it's famously cited as one of computer science's hardest problems:

• TTL (Time To Live) expiration — the simplest approach: cache entries automatically expire after a fixed duration, bounding how stale data can get, at the cost of the data being potentially stale for up to that entire duration.
• Explicit invalidation on write — when data changes, actively delete (or update) its corresponding cache entry, rather than waiting for a TTL to expire:

def update_product_price(product_id, new_price_cents):
    db.execute("UPDATE products SET price_cents = %s WHERE id = %s", [new_price_cents, product_id])
    redis_client.delete(f"product:{product_id}")  # invalidate -- the NEXT read will re-fetch fresh data

• Event-driven invalidation — for systems with multiple services or cache instances, publishing an event ("product 42 changed") that all relevant caches subscribe to and react to by invalidating their own copy — directly the Observer pattern (companion Design Patterns book, Chapter 9) and Event-Driven Architecture (that book's Chapter 16), applied specifically to cache coherence across multiple, independently-running cache instances.

Choosing a TTL: a real, consequential trade-off

[IN PRACTICE] A short TTL means data is rarely stale for long, but the cache provides less protection against database load (frequent re-fetching on expiration) and increases the frequency of thundering-herd risk on popular keys. A long TTL means much better database load protection and fewer thundering-herd events, but data can be stale for a correspondingly longer window — the right choice depends entirely on how tolerant the specific data is of staleness: a product's price probably needs a short TTL or explicit invalidation (customers seeing a stale price is a real business problem), while a product's description (which changes rarely) can tolerate a much longer TTL with little practical downside.

Common mistakes

1. Caching data with no invalidation strategy at all beyond "it'll expire eventually," for data where staleness is actually a meaningful, visible business problem (prices, inventory counts, account balances) — TTL-only invalidation is appropriate for tolerant data, not universally.
2. Using write-behind caching as a default "fast writes" pattern without explicitly accepting (and communicating to stakeholders) the real data-loss risk it introduces on a crash — this should be a deliberate, narrow, well-understood trade-off, never an unexamined default.
3. Not protecting against thundering herd on popular cache keys — assuming cache misses are independent, rare events rather than potentially simultaneous, highly correlated ones under real production load (directly, the same pattern covered in the companion OOP book's Chapter 14 and Networking book's Chapter 11).
4. Caching data that changes on nearly every read anyway, gaining little to no benefit while adding invalidation complexity for no real payoff — caching pays off specifically when reads significantly outnumber writes for the cached data.

Checklist: designing a caching layer

1. Choose cache-aside as the default pattern unless a specific requirement (always-fresh cache, or extreme write-latency sensitivity) points toward write-through or write-behind specifically.
2. Match TTL/invalidation strategy to the data's actual staleness tolerance — short TTL or explicit invalidation for business-critical, frequently-changing data; longer TTL for slowly-changing, staleness-tolerant data.
3. Always use explicit invalidation (not just TTL) for data where staleness has a real, visible business cost (prices, inventory, account balances).
4. Protect popular cache keys against thundering herd with a single-flight/locking pattern on cache-miss regeneration.
5. Only adopt write-behind caching with an explicit, documented acceptance of its data-loss risk — never as an unexamined default for "faster writes."

Discussion / interview questions

1. "What's the difference between cache-aside, write-through, and write-behind caching?" Strong answer: Cache-aside: application checks cache first, falls through to the database on a miss and populates the cache. Write-through: every write updates both cache and database together, keeping the cache always current. Write-behind: writes go to the cache immediately and are asynchronously flushed to the database later, trading durability (real data-loss risk on a crash) for write speed.

2. "Why is cache invalidation considered one of the hardest problems in computer science?" Strong answer: There's no universally correct strategy — TTL-based expiration is simple but allows staleness for its whole duration; explicit invalidation requires tracking every place data can change and reliably triggering invalidation; event-driven invalidation across multiple services/cache instances adds real distributed-systems complexity — every approach involves a genuine trade-off, not a clean, obviously-correct solution.

3. "How would you decide on an appropriate TTL for a cached value?" Strong answer: Bases it on the data's actual staleness tolerance and business impact — a frequently-changing, business-critical value (a price) needs a short TTL or explicit invalidation; a rarely-changing, staleness-tolerant value (a product description) can use a much longer TTL, trading some staleness risk for significantly better database load protection.

4. "What's the risk of write-behind caching, and when might it still be an acceptable trade-off?" Strong answer: A crash between the cache write and the asynchronous database flush loses that write entirely, since it never reached durable storage; acceptable only in specific, narrow, high-write-throughput scenarios where this data-loss risk has been explicitly, deliberately accepted as a known trade-off — never as an unexamined default.

Chapter summary / cheat sheet

• Cache-aside: check cache, fall through to database on miss, populate cache — the most common default pattern, but the first request for a key always pays a "cold" cost.
• Write-through: cache and database updated together — always current, at extra write latency.
• Write-behind: cache updated immediately, database flushed asynchronously — fastest writes, real data-loss risk on crash; only for deliberate, narrow use cases.
• Invalidation is the genuinely hard problem: TTL expiration (simple, bounded staleness), explicit invalidation on write (precise, requires tracking every write path), event-driven invalidation (needed across multiple services/cache instances — directly the Observer/Event-Driven Architecture patterns applied to cache coherence).
• Always protect popular cache keys against thundering herd; match TTL/invalidation strategy to the specific data's actual staleness tolerance, not a one-size-fits-all default.

---