AI coding agents raise the lines of code a company generates by 30%, but not the number of features it ships. That is the finding of a study by Fiona Chen and James Stratton, Harvard University researchers who analyzed data from more than 700 software firms. The reason is code review: the time between opening a pull request (a request to add new code to a project) and approving it went up 49% after agents arrived.
The Ars Technica article, by Kyle Orland, summarizes the study and is where every number in this text comes from. If you hire developers, an agency or AI tools to run your business site, the count that matters is not how much code comes out. It is how much code reaches production.
Who was studied and how did the study measure the effect of AI?
The study used aggregated data from Jellyfish, a platform that measures engineering team output. The dataset covers 300 million “work events” (commits, pull requests and the like) and task-management data for more than 700,000 employees at more than 700 software firms, from 2021 to March 2026.
To find out when each firm started using AI, the researchers combined direct usage measurement with analysis of GitHub activity. They separated two kinds of tool: assistants, which complete code written mostly by humans, and agents, which write and submit code on their own from a prompt. Then they compared the same metrics before and after adoption, firm by firm, using a method called “difference of differences”.
What changes when a company adopts coding agents?
Raw output goes up, and by a lot. According to the study, the arrival of agents is linked to more code, more commits and more pull requests, but the resolution rate of tasks in Jira (the team’s task-management tool) does not change in a statistically significant way.
| Metric | Effect after agents |
|---|---|
| Lines of code generated | +30% |
| Commits | +20% |
| Pull requests | +23% |
| Review time (from submission to merge) | +49% |
| Comments per pull request | +35% |
| Pull requests with changes requested | nearly doubled |
| Employees doing code review | +14% |
| Issue and epic resolution (Jira) | no significant change |
The researchers also found no change in the size or complexity of the tasks tracked in Jira, which weakens the explanation that firms started tackling smaller problems. On employment, they wrote that they “cannot attribute significant employment changes” to AI after cross-referencing Jellyfish data with LinkedIn.
Why doesn’t more code turn into more software?
Because agent-written code has to be read by a person before it goes live, and that queue did not keep up with the volume. In the authors’ words, the gain in the writing phase is “absorbed by downstream constraints in the production process”. More pull requests arrive, they come back more often with change requests, and they collect more comments.
I worked in banking and learned an early rule about process: the speed of the whole is the speed of the slowest step. Speeding up typing without touching review is filling a funnel that still has the same bottleneck.
AI could also help with review, but so far it has helped little. By March 2026, 80% of the measured firms already used some form of AI code review. Even so, agents accounted for 23.3% of review comments and 10.8% of pull requests, which indicates that most of the work still sat with humans.
Does the study prove coding agents are not worth it?
No. The Ars Technica piece itself adds the caveat: the data ends in March 2026, agents have improved since, and 95% of the firms in the study have now adopted agents and are still learning when to use them. The balance between writing time and review time may shift as teams gain practice.
What the study shows is that adopting agents without changing the review process does not deliver more features. That is different from saying the tool is no good. Ars Technica goes as far as asking whether the time and expense of getting agents to work pays off for most companies, and that is the publication’s judgment, not a result measured by the study.
What should you check on Monday if you pay for AI-assisted development?
Swap the question “how much code was produced?” for “how much was delivered, and how long did it take to be approved?”. Ask your team or vendor for three numbers from the last quarter:
- Average time between opening and approving a pull request. If it rose after AI came in, the bottleneck is review.
- Share of pull requests that come back with change requests. In the study, it nearly doubled.
- Tasks completed in the same period, in Jira or whatever tool the team uses. That is the number your business feels.
If a vendor shows only lines of code or commit counts, they are reporting the metric the study saw grow with no effect on deliveries. The same goes for an agency that uses AI to build pages and components for your site: ask who reviews and how long that takes.
Cost enters the picture too. An agent that generates a lot of code burns a lot of tokens, and measuring that spend matters as much as measuring delivery. Anyone following what platforms are building for agents, like Cloudflare’s Clef and Kitesurf, sees the same movement: infrastructure for agents got cheaper and more accessible, and verifying what they produce is still human work.
What does the study mean, in short, for decision makers?
Across 700 software firms, AI coding agents raised code output by roughly 20% to 30% and did not change the task resolution rate. Review got 49% slower, with nearly double the pull requests needing changes and 35% more comments per pull request. AI review exists at 80% of firms but covers a small share of the work.
The practical decision: before buying more agents, measure your team’s review capacity. If it does not grow with code volume, the investment shows up in the commit report and not in the product. The study runs only to March 2026, so repeat this measurement on your team every quarter instead of treating its result as final.