Chapter 11: Measuring Delivery Health: Cycle Time, Deployment Frequency, Change Failure Rate, MTTR

Why this matters

"Are we shipping software well?" is a question teams often answer with gut feeling or anecdote, when four well-established, measurable metrics (popularized by the DORA — DevOps Research and Assessment — research program) provide a genuinely evidence-based answer. These four metrics, tracked together, give a balanced picture of both delivery speed and delivery stability — deliberately as a set, since optimizing any one in isolation can produce a misleading, even counterproductive, picture.

The four key metrics

[BEGINNER]

1. Deployment frequency — how often code is deployed to production. High-performing teams deploy many times per day; lower-performing teams might deploy monthly or less. Directly connects to Chapter 9's argument for frequent, small, low-risk deployments over infrequent, large ones.
2. Lead time for changes (cycle time) — how long it takes from a code commit to that code running in production. Shorter lead time means faster feedback and faster value delivery.
3. Change failure rate — what percentage of deployments cause a production failure requiring remediation (a rollback, a hotfix, an incident). Lower is better — this is the metric that keeps deployment frequency and lead time honest, preventing a team from "winning" on speed by sacrificing stability.
4. Mean time to recovery (MTTR) — when a production failure does occur, how long it takes to restore service. Directly connects to Chapter 10's incident response practices — a fast, structured mitigation process (rollback, feature flag toggle) produces a lower MTTR than an ad hoc, improvised response.

Why these four, specifically, and why together

[SENIOR INSIGHT] These four metrics are deliberately chosen as a balanced set spanning both speed (deployment frequency, lead time) and stability (change failure rate, MTTR) — this pairing is the entire point, and it directly counters a common, damaging misconception: that speed and stability are opposing forces you must trade off against each other. DORA's own research finding (widely replicated) is that high-performing teams excel at all four simultaneously — teams that deploy frequently, with short lead times, actually tend to have lower change failure rates and faster recovery times too, not higher ones, because the same practices that enable speed (small changes, strong automation, good monitoring — directly this book's Chapters 6, 9, and 10) are exactly the practices that also enable stability. Slow, infrequent, large-batch releases don't buy safety; they tend to produce worse outcomes on both dimensions simultaneously.

Common misconception: "We deploy slowly and carefully, which makes us
more STABLE, even if we're not as fast as teams that deploy constantly."

What the research actually shows: teams making this trade-off typically
have WORSE change failure rates too, not better -- because slow,
infrequent, large-batch deployments accumulate more changes per release
(harder to reason about, per Chapter 9), and the muscle of "deploying
safely" atrophies with infrequent practice, rather than being sharpened.

Using these metrics without falling into Chapter 12-style (companion Code Reviews book) metric-weaponization

[IN PRACTICE] Directly the same caution the companion Code Reviews book raised for review metrics: these four DORA metrics are meant to be tracked at the team or organization level, to identify systemic bottlenecks and track genuine improvement over time — not to individually rank or pressure specific engineers. "Our lead time has grown from 2 hours to 3 days over six months" is a legitimate, actionable team-level signal worth investigating (is testing too slow? is the deployment pipeline too manual? are PRs too large, per Chapter 6?). "Engineer X's commits take longer to reach production than Engineer Y's" is exactly the kind of individual-level metric misuse likely to produce Goodhart's Law gaming rather than genuine improvement — and it also ignores that lead time is heavily influenced by the surrounding system (CI speed, review availability, deployment process) far more than any individual's own behavior.

Common mistakes

1. Optimizing deployment frequency or lead time without tracking change failure rate/MTTR alongside them, risking a false sense of improvement that's actually just shipping faster while quietly becoming less stable.
2. Believing slow, infrequent releases are inherently safer, when the evidence generally shows the opposite — the practices enabling speed (small changes, strong automation) are largely the same practices enabling stability.
3. Using these metrics to individually rank or pressure specific engineers, rather than tracking them in aggregate to identify systemic bottlenecks worth addressing as a team.
4. Not investigating a concerning metric trend's root cause before reacting to it, similar to the companion Code Reviews book's Chapter 12 guidance — a metric should start an investigation, not immediately justify a unilateral mandate.

Checklist: using delivery health metrics well

1. Track deployment frequency, lead time, change failure rate, and MTTR together, as a balanced set spanning speed and stability — never optimize one in isolation.
2. Investigate concerning trends at the systemic level (pipeline speed, PR size, review capacity, incident response process) rather than reacting with an unexamined mandate or blaming individuals.
3. Never use these metrics to individually rank or evaluate specific engineers — they're team/organization-level signals, heavily influenced by surrounding systems and process, not individual output measures.
4. Expect (and look for) all four metrics to improve together as underlying practices improve — if speed improves while stability degrades, that's a signal something in the underlying practice (not just the pace) needs attention.

Discussion / interview questions

1. "What are the four key DORA metrics, and why are they tracked together rather than individually?" Strong answer: Deployment frequency and lead time for changes (speed), change failure rate and MTTR (stability) — tracked together because they counter the misconception that speed and stability trade off against each other; high performers tend to excel at all four simultaneously, since the same practices (small changes, automation, monitoring) enable both.

2. "Does deploying more slowly and infrequently generally make a team more stable?" Strong answer: No — evidence generally shows the opposite: slow, infrequent, large-batch releases tend to have worse change failure rates too, since larger batches are harder to reason about and the practice of deploying safely atrophies with infrequent use rather than improving.

3. "Why should deployment frequency and lead time never be optimized without also tracking change failure rate and MTTR?" Strong answer: Optimizing speed alone risks a false sense of improvement — a team could be shipping faster while quietly becoming less stable; tracking all four together ensures speed improvements aren't coming at stability's expense.

4. "Would you use these metrics to compare individual engineers' performance?" Strong answer: No — these are team/organization-level signals heavily influenced by surrounding systems (CI speed, review capacity, deployment process) rather than individual behavior; using them to rank individuals would likely produce Goodhart's Law-style gaming and misses that the underlying system, not individual effort alone, drives these numbers.

Chapter summary / cheat sheet

• Four key metrics (from DORA research): deployment frequency and lead time for changes (speed); change failure rate and MTTR (stability) — tracked together as a balanced set.
• High-performing teams excel at all four simultaneously — speed and stability aren't opposing forces; the same practices (small changes, automation, monitoring) enable both.
• Track these at the team/organization level to identify systemic bottlenecks — never to individually rank or pressure specific engineers, echoing the companion Code Reviews book's metric-weaponization warning.
• A concerning trend should prompt investigation into root cause and underlying practice, not an unexamined mandate or individual blame.

---