What changed
Most coding-assistant claims come from short lab tasks or company anecdotes. This paper followed three real workplace experiments instead. Microsoft, Accenture, and an anonymous Fortune 100 electronics manufacturer each randomized when software developers could access GitHub Copilot, then measured what happened inside normal engineering workflows over periods lasting two to eight months.
The paper's main output measure was completed pull requests, which function as a practical unit of finished software work inside these organizations. Pooling the three experiments with the authors' preferred instrumental-variable approach, developers who used Copilot completed 26.08% more pull requests per week. Secondary measures moved in the same direction: weekly commits increased 13.55% and weekly builds increased 38.38%.
The most useful heterogeneity result came from Microsoft, where the researchers could observe tenure and job level. Short-tenure developers increased output by 27% to 39%, versus 8% to 13% for longer-tenure developers. Junior developers increased output by 21% to 40%, versus 7% to 16% for senior developers. Adoption also started slowly until reminder emails and training nudges increased take-up, which matters because a tool cannot help workers who never really try it.
What this could change for you
For software teams, the believable use case is not autonomous engineering. It is moving more routine coding work through the queue faster: scaffolding code, writing boilerplate, suggesting completions, and helping a developer reach a reviewable pull request sooner. That can matter when backlogs are large and senior review time is scarce.
For managers, the study argues for measuring workflow output rather than chatbot excitement. The experiments were embedded in normal work, and the strongest effects showed up in concrete engineering traces such as pull requests and commits. But the adoption data also show that rollout details matter. Microsoft needed reminder emails, and Accenture paired access with training and manager encouragement.
For workers, the result is a reminder that AI gains do not have to mean replacing the most experienced people first. In these experiments, newer and more junior developers benefited most. That makes coding assistants look more like onboarding and throughput tools than like a system that erases the need for senior judgment, code review, or architectural thinking.
What it does not prove
This was one tool in one occupation. The evidence says something about GitHub Copilot inside three software-development organizations, not about marketing, finance, design, legal work, or every other white-collar task now being bundled into "AI at work."
The paper measures workflow output, not downstream software value. More pull requests, commits, and builds do not automatically mean better product decisions, fewer bugs in production, stronger security, or more maintainable code. The study found no clear overall improvement in build success rate, and one experiment at Accenture pointed in a negative direction on that quality proxy.
The experiments were noisy and operationally imperfect. Adoption was incomplete, one company used a staggered rollout rather than a simple treatment-control split, and one additional Accenture experiment was excluded from the main table because layoffs damaged the data. The headline result is credible because it comes from randomized workplace evidence, but it is still an estimate built from heterogeneous real-world rollouts rather than a clean universal law of software engineering.
The bottom line
This is one of the strongest low-risk AI-at-work results so far: a coding assistant increased completed software tasks across three randomized workplace experiments, with the biggest gains among newer and more junior developers. The honest takeaway is narrower than the hype. Copilot did not prove it can replace engineers or guarantee better software. It helped developers ship more reviewable work inside ordinary workflows where humans still judged what to keep.
Primary research
The Effects of Generative AI on High-Skilled Work: Evidence from Three Field Experiments with Software Developers
Management Science · 2026 · DOI 10.1287/mnsc.2025.00535
