A director once asked me to report lines of code per developer per week. I said no, and offered to report something useful instead. That conversation happens more often now, because AI coding tools have made output volume shoot up while nobody is quite sure whether anything got better.
It is a fair question badly framed. Something did change on my team when we adopted these tools. Pull requests got about 40% larger. Review time went up, not down. And the number that actually mattered — how long it took an idea to reach a customer — barely moved for the first few months.
Measuring that properly is what DORA is for, and it has become more useful in the AI era rather than less, precisely because the naive metrics have become so misleading.
The Four Metrics, and Why These Four
Deployment frequency — how often you ship to production.
Lead time for changes — commit to running in production.
Change failure rate — what share of deploys cause a problem needing a fix or rollback.
Time to restore service — how long recovery takes when something breaks.
What makes these hold up where other metrics do not is the pairing. The first two measure speed, the second two measure stability, and you cannot cheat one pair without wrecking the other. Ship recklessly to raise deployment frequency and your change failure rate exposes it. Freeze everything to protect stability and your lead time collapses.
Compare that to lines of code, story points or commits — all of which an individual can inflate on a bad afternoon without producing anything.
They are also team metrics, not individual ones. The moment you put lead time on a person's review, you have created an incentive to split work artificially and avoid hard problems. I have watched that happen and it took a year to undo the damage to how people chose what to work on.
What AI Tools Actually Changed
Being specific, from watching the numbers rather than the marketing.
Code production stopped being the bottleneck. It mostly was not the bottleneck before, which is why raw output going up did not move delivery. The constraint moved to review, verification and deployment — and if those are unchanged, faster typing produces a bigger queue, not faster delivery.
Lead time can get worse before it gets better. Ours did. Larger PRs sat longer in review. A 600-line PR does not get reviewed in the same way a 150-line one does; it gets skimmed, or it waits for someone with an hour free.
Change failure rate is the metric to watch closely. Generated code is fluent, which disables the instinct reviewers rely on. Subtly wrong code that looks competent gets through more easily than obviously junior code. If your change failure rate is creeping up while everything feels more productive, that is the signal.
Restore time barely moves. AI helps with diagnosis a little. It does not help with rollback discipline, on-call clarity or whether you have tracing.
The teams I have seen genuinely get faster with these tools are the ones that also invested in the constraint — smaller PRs, better tests, faster CI, safer deploys. The tool removed the typing; the pipeline had to remove the waiting.
Measuring Without a Platform
You do not need a vendor. Three of the four come out of git and your CI system.
-- Lead time: first commit on a branch → deployed to production.
SELECT
date_trunc('week', d.deployed_at) AS week,
percentile_cont(0.5) WITHIN GROUP (ORDER BY
EXTRACT(EPOCH FROM d.deployed_at - c.first_commit_at) / 3600) AS median_hours,
percentile_cont(0.9) WITHIN GROUP (ORDER BY
EXTRACT(EPOCH FROM d.deployed_at - c.first_commit_at) / 3600) AS p90_hours,
count(*) AS deploys
FROM deployments d
JOIN pull_requests c ON c.merge_sha = d.sha
WHERE d.environment = 'production' AND d.deployed_at > now() - interval '90 days'
GROUP BY 1 ORDER BY 1;
Track the median and the p90 together. The median tells you the normal case; the p90 tells you about the work that gets stuck, which is usually where the real problem lives. An average hides both.
Change failure rate needs one piece of discipline: tag the deploys that caused an incident. A label on the incident linking the deploy is enough. Restore time comes from your incident records — start to resolution.
The Metrics I Added Alongside
DORA measures the delivery pipeline. It says nothing about whether the work was worth doing or whether the team can sustain it. Four more I have found worth tracking:
PR size distribution. The leading indicator for everything else. When median PR size grows, review time and change failure rate follow within a few weeks. This became the most actionable number on my team after we adopted AI tools.
Review wait time, separated from review duration. "Waiting for a reviewer" and "being reviewed" are different problems with different fixes — one is scheduling, the other is complexity.
CI duration, p90. Past about ten minutes people stop waiting and start context-switching, and your feedback loop is broken regardless of what the pipeline reports.
Rework rate — code changed again within two weeks of shipping. A rising number means things are going out before they are right, which is what a change failure rate can miss when the fix is small enough not to count as an incident.
How These Get Misused
Turning them into targets. Any metric that becomes a target stops measuring what it measured. Deployment frequency as an OKR produces empty deploys. Use them to find your constraint, not to grade anyone.
Comparing teams. A team on a legacy monolith with a compliance gate is not comparable to one shipping a greenfield service. Compare a team to its own trend.
Chasing "elite" tiers. Multiple deploys a day is genuinely valuable for a consumer product. For firmware or a regulated system, it is not the right goal, and pretending otherwise wastes everyone's time.
Measuring instead of asking. The numbers show you where the delay is. They never tell you why. That still comes from talking to the people doing the work.
What I Would Track First
If you measure nothing today, start with two: lead time p90 and change failure rate. One tells you whether work is flowing, the other whether you are trading safety for it. Both come from data you already have, and together they catch the trap AI tooling sets — feeling much faster while shipping more defects.
For that director, the number we ended up reporting was median lead time by week. It made a much better conversation, because when it went the wrong way we could point at review capacity rather than at people.
