Chapter 4: Back-of-Envelope Capacity Estimation

Why this matters

Elicited requirements (Chapter 3) like "10 million daily active users" don't directly tell you whether a single database server can handle the load, or whether you need sharding — converting user-level numbers into concrete, actionable figures (requests per second, storage growth, bandwidth) is what makes a requirement actually usable for architecture decisions, rather than an impressive-sounding but practically inert number.

Converting daily active users into requests per second

[BEGINNER] The standard technique: estimate actions per user per day, multiply by daily active users, then divide by the number of seconds in a day — while accounting for the fact that traffic isn't evenly distributed across the day.

Given: 10 million daily active users, each performing ~20 actions/day on average.

Total daily actions = 10,000,000 x 20 = 200,000,000 actions/day

Seconds per day = 86,400

Average requests/second = 200,000,000 / 86,400 ≈ 2,315 req/s

PEAK traffic is NOT evenly distributed -- a common rule of thumb assumes
peak traffic is roughly 2-3x the average (accounting for daily usage
patterns, like an evening peak):

Peak requests/second ≈ 2,315 x 3 ≈ 6,945 req/s

This peak number, not the average, is what capacity planning should
design around -- a system that only handles average load will fall
over precisely when it matters most (at peak usage).

Estimating storage growth

[BEGINNER]

Given: 200,000,000 actions/day, each generating a record of ~500 bytes.

Daily storage growth = 200,000,000 x 500 bytes = 100,000,000,000 bytes
                       = 100 GB/day

Over 1 year: 100 GB x 365 ≈ 36.5 TB/year

This number directly informs decisions later in this book: does a
SINGLE database server's storage capacity suffice, or does this scale
require sharding (companion Database book, Chapter 11) within a
realistic planning horizon (e.g., 2-3 years)?

Using memory hierarchy latency numbers for a latency budget

[SENIOR INSIGHT] The companion Operating System book's Chapter 3 memory hierarchy latency numbers are directly reusable here for reasoning about where a request's time actually goes: if a request's total latency budget is 200ms, and a single database round-trip is ~5-10ms, a single cache lookup is <1ms, and a cross-region network round-trip is 100-300ms (companion Networking book, Chapter 9), these numbers immediately reveal which operations a design can afford and which it can't. A design requiring three sequential cross-region network round-trips within a 200ms budget is immediately, quantifiably infeasible — no amount of clever code changes that arithmetic, and recognizing this from the latency numbers alone (rather than discovering it after building the system) is exactly the value back-of-envelope estimation provides.

Latency budget: 200ms total for a user-facing request.

Rough allocation:
- Network round-trip (client to server, same region): ~20-50ms
- Cache lookup: <1ms
- Database query (if cache miss): ~5-20ms
- Application processing: ~10-30ms
- Remaining buffer for variability/tail latency: the rest

If the design requires a call to another service in a DIFFERENT region
(100-300ms round trip ALONE), the 200ms budget is already blown before
any actual processing happens -- this single estimation immediately
rules out that specific design choice, without needing to build
anything to discover the problem.

Why precision doesn't matter, but the right order of magnitude does

[IN PRACTICE] Back-of-envelope estimation is explicitly not about precise accuracy — nobody expects (or should expect) an estimate like "6,945 req/s" to be exactly right; what matters is getting the right order of magnitude, since that's what actually determines architectural decisions. The difference between 6,945 req/s and 8,000 req/s changes nothing about the architecture; the difference between 6,945 req/s and 6,945,000 req/s (three orders of magnitude) changes everything — a single well-provisioned server might handle the former, while the latter unambiguously requires horizontal scaling and load balancing (Chapter 6) from the start. This is why rounding aggressively during estimation (using round numbers like "roughly 20 actions per day" rather than a precise, unfounded "23.7 actions per day") is not just acceptable but the correct practice — it correctly signals that the estimate's value is in its magnitude, not its false precision.

Common mistakes

1. Treating a back-of-envelope estimate as needing to be precisely accurate, spending disproportionate time refining a number whose actual purpose is establishing the right order of magnitude, not exact accuracy.
2. Estimating only average load and designing capacity around it, missing that peak load (commonly 2-3x average) is what a system actually needs to handle without falling over.
3. Not connecting latency estimates to a stated latency budget (Chapter 3) to check design feasibility early — this is precisely the calculation that can rule out an infeasible design (like multiple sequential cross-region calls) before any implementation effort is spent discovering the same problem the hard way.
4. Skipping storage growth estimation, missing an early, cheap signal about whether sharding or other scaling strategies need to be part of the initial design versus a reasonable future consideration.

Checklist: performing back-of-envelope estimation effectively

1. Convert user-level numbers (daily active users) into requests per second, accounting for peak-vs-average traffic (a common rule of thumb: peak ≈ 2-3x average).
2. Estimate storage growth over a realistic planning horizon (1-3 years) to determine whether single-node storage capacity suffices or scaling strategies need early consideration.
3. Use known latency figures (network round-trips, database queries, cache lookups) to check whether a proposed design fits within a stated latency budget, ruling out infeasible designs early and cheaply.
4. Round aggressively and focus on order-of-magnitude correctness, rather than pursuing false precision in numbers whose actual purpose is establishing rough architectural scale.

Discussion / interview questions

1. "How would you estimate the peak requests-per-second a system needs to handle, given a daily active user count?" Strong answer: Estimate actions per user per day, multiply by daily active users, divide by seconds per day to get average req/s, then multiply by a peak factor (commonly 2-3x) to account for uneven daily traffic distribution — designing capacity around the peak number, not the average.

2. "Why is precision unimportant in back-of-envelope estimation, while order of magnitude is critical?" Strong answer: Architectural decisions (single server vs. horizontal scaling, whether sharding is needed) are determined by order of magnitude, not precise figures — the difference between similar-magnitude estimates changes nothing architecturally, while an order-of-magnitude difference changes everything, so aggressive rounding is correct practice, not sloppiness.

3. "How would you use latency estimation to check whether a proposed design is even feasible, before building anything?" Strong answer: Sum the estimated latency of each step in the request path (network round-trips, database queries, cache lookups) using known reference figures, and compare against the stated latency budget — a design requiring, say, multiple sequential cross-region calls can be shown infeasible purely through this arithmetic, without needing to build and measure it.

4. "Why should storage growth be estimated even early in a design conversation?" Strong answer: It's a cheap, early signal for whether single-node storage capacity will suffice over a realistic planning horizon or whether sharding/partitioning needs to be part of the initial design consideration — discovering a storage capacity problem after launch is far more expensive than anticipating it during design.

Chapter summary / cheat sheet

• Convert user-level numbers into requests per second: actions/user/day × daily active users ÷ seconds/day = average req/s; multiply by a peak factor (commonly 2-3x) for the number that actually matters for capacity planning.
• Estimate storage growth over a realistic planning horizon to catch an early signal about whether sharding needs to be part of the initial design.
• Use known latency reference figures (network round-trips, database queries, cache lookups — many from the companion OS and Networking books) to check a proposed design's feasibility against a stated latency budget, before building anything.
• Precision doesn't matter; order of magnitude does — round aggressively, since architectural decisions are determined by magnitude, not exact figures.

---