Chapter 11: Capacity Planning and Load Testing

Why this matters

Chapter 4 established that capacity shortfalls are reliability failures. This chapter covers the specific practice of proving a system's actual capacity through load testing, rather than relying purely on the companion System Design book's Chapter 4 back-of-envelope estimation — estimation tells you the right order of magnitude to design for; load testing tells you whether the actual, built system genuinely achieves it.

Why estimation alone isn't sufficient verification

[SENIOR INSIGHT] Back-of-envelope estimation (System Design book, Chapter 4) is essential for early design decisions, but it's built on assumptions (about per-request cost, about how components will actually perform together, about real-world data distribution) that can be wrong in ways that only become visible once the actual system exists and can be measured directly. Load testing closes this gap by measuring the real system's real behavior under real (simulated) load, rather than continuing to reason from assumptions alone once the system is built and available to test.

Types of load testing, and what each verifies

[BEGINNER]

• Load testing — verify the system performs correctly at its expected, normal peak load (the number estimation, Chapter 4, produced).
• Stress testing — push load beyond expected peak, specifically to find the actual breaking point and observe how the system fails (gracefully, per Chapter 7's load shedding, or catastrophically) — knowing your system's actual limit, and its failure behavior at that limit, is valuable even though you don't expect to operate there routinely.
• Soak testing — sustain a realistic load for an extended period (hours, not minutes), specifically to catch problems that only manifest over time — a slow memory leak, gradual connection pool exhaustion, or a resource that's never properly released, none of which a short load test would reveal.

Load test: "does the system handle our estimated 8,000 req/s peak
            correctly, for a normal test duration (e.g., 15-30 minutes)?"

Stress test: "at what load does the system ACTUALLY start failing, and
              does it fail gracefully (load shedding, per Chapter 7) or
              catastrophically (a full crash affecting everyone)?"

Soak test: "does the system remain healthy handling a realistic,
            sustained load for 12+ HOURS, or does something degrade
            slowly over that time that a 30-minute test would never reveal?"

Common capacity planning mistakes load testing specifically catches

[IN PRACTICE] Load testing frequently reveals problems that pure estimation and code review miss entirely: a connection pool (Chapter 6) sized incorrectly only becomes visible under genuinely concurrent load; a cache stampede (Chapter 5) may only manifest with enough genuinely concurrent cache-miss requests; a slow memory leak is invisible in a quick manual test but obvious after a multi-hour soak test. This is precisely why load testing is a distinct, necessary practice rather than something estimation and careful code review alone can substitute for.

Testing with realistic data and traffic patterns

[SENIOR INSIGHT] A load test's value depends heavily on how realistic its simulated traffic actually is — testing with uniform, evenly-distributed synthetic requests when real traffic is actually bursty and unevenly distributed (a small number of very popular items receiving a disproportionate share of requests, directly Chapter 5's cache stampede risk) can produce a false sense of confidence: the system "passes" a load test that doesn't actually resemble the traffic pattern that would cause a real problem. Effective load testing uses production traffic patterns (replayed, anonymized real traffic logs where feasible) or, at minimum, a deliberately realistic traffic distribution model, rather than convenient-to-generate but unrealistic uniform synthetic load.

Common mistakes

1. Relying purely on back-of-envelope estimation without ever load testing the actual, built system, missing problems that only become visible when the real system is measured under real simulated load.
2. Only load testing at expected peak, never stress testing beyond it, missing the chance to understand the system's actual breaking point and failure behavior before a real, unplanned event reveals it under worse conditions.
3. Never soak testing, missing slow-developing problems (memory leaks, gradual resource exhaustion) that only manifest over an extended, sustained load period.
4. Load testing with unrealistic, uniformly-distributed synthetic traffic when real traffic is bursty and unevenly distributed, producing a false sense of confidence that doesn't reflect the actual conditions likely to cause a real problem.

Discussion / interview questions

1. "Why isn't back-of-envelope capacity estimation sufficient on its own, without load testing the actual system?" Strong answer: Estimation relies on assumptions (per-request cost, component interaction behavior, data distribution) that can be wrong in ways only visible once the real system is measured directly — load testing verifies the actual, built system's real behavior rather than continuing to reason from assumptions alone.

2. "What's the difference between load testing, stress testing, and soak testing?" Strong answer: Load testing verifies correct behavior at expected peak load; stress testing pushes beyond peak to find the actual breaking point and observe failure behavior; soak testing sustains realistic load over an extended period to catch slow-developing problems (memory leaks, gradual resource exhaustion) that shorter tests can't reveal.

3. "Why might a system 'pass' a load test but still fail under real production traffic?" Strong answer: If the load test used unrealistic, uniformly-distributed synthetic traffic while real traffic is bursty and unevenly distributed (a small number of very popular items receiving disproportionate load), the test doesn't actually exercise the failure modes (like cache stampede) that real, uneven traffic patterns would trigger — producing false confidence.

4. "What kind of problem would a soak test catch that a standard 30-minute load test wouldn't?" Strong answer: Slow-developing issues — a gradual memory leak, connection pool exhaustion accumulating over time, or a resource that's never properly released — that only become visible after sustained load over hours, invisible in a short test window regardless of how much peak load it applies.

Chapter summary / cheat sheet

• Back-of-envelope estimation (companion System Design book, Chapter 4) tells you the right order of magnitude to design for; load testing verifies whether the actual, built system genuinely achieves it — the two are complementary, not substitutes.
• Load testing (expected peak), stress testing (beyond peak, to find the breaking point and failure behavior), and soak testing (sustained load over hours, to catch slow-developing problems) each verify something the others don't.
• Load testing frequently catches problems (connection pool sizing, cache stampede, memory leaks) that pure estimation and code review miss entirely.
• Use realistic, bursty/unevenly-distributed traffic patterns in load tests, not convenient uniform synthetic load — an unrealistic test can produce false confidence that doesn't reflect real production risk.

---