AI Productivity Is Not Proof of Better Work

What productivity evidence actually establishes

Productivity evidence shows how much defined output was produced for an input, or how long a specified result took. The OECD’s Measuring Productivity guidance frames the measure as a relationship between outputs and inputs. In a task study, the numerator might be accepted reports completed and the denominator worker time. A time comparison asks how many minutes were needed for the same report. “Faster” is incomplete unless output and input are clear. “Accepted” matters: generated drafts or cases marked closed may not equal finished work if someone must correct errors or complete missing steps. Capability benchmarks show what a system can do on a test; user satisfaction records how people feel about using it. Neither directly measures workplace productivity. A task-level gain does not establish that the result was accurate, useful, safe, or better for its recipient. Nor does it establish employer adoption or job loss. When a study reports higher productivity, ask what it counted before applying the result to another task.

Sources: Measuring Productivity

What additional evidence makes a case for better work?

A better-work claim needs evidence about the result, not only how quickly it was produced. Compare accepted output per unit of total effort with whether that output met a task-relevant quality standard: accuracy, completeness, clarity, safety, or usefulness to the next person in the process. Set the standard before comparing assisted and unassisted work; otherwise, a faster process can lower the bar and still appear to improve. Quality has layers. Reviewers may rate an output as clearer or more useful, yet it can still fail when someone relies on it. A polished draft may contain a factual error; a case marked complete may leave the customer’s issue unresolved. These are different findings: assessed output quality describes an artifact under a stated review method, while a downstream service or customer outcome describes what happened after the work entered a process. Neither substitutes for the other. Safety-sensitive work also requires attention to serious exceptions, not just average ratings across routine cases. “Experimental evidence on the productivity effects of generative artificial intelligence” shows that a study can measure both speed and judged quality. In a preregistered online experiment, 453 college-educated professionals completed occupation-specific, incentivized writing tasks; half were randomly given access to ChatGPT. The abstract reports average completion time falling 40% and output-quality ratings rising 18%. This supports a joint gain on those tested writing tasks. It does not establish better customer decisions, fewer later corrections, or improvement in every professional workflow. The quality result matters, but its meaning stays attached to the task and rating method the study reports. A useful quality measure names who judged the output, the rubric or acceptance threshold, and whether reviewers knew which condition produced it. It should identify error types and severity, exceptions requiring escalation, and any downstream check tied to the task. For customer support, distinguish a case closed in a system from a problem resolved; for a report draft, check factual claims against source material. These are examples of measurement choices, not findings about a particular workplace. The strongest evidence combines a fair productivity comparison with independent quality assessment and an outcome close to the task’s real consequence. If only one layer is measured, state that boundary: faster drafts establish speed; higher reviewer ratings establish judged quality under that review; resolved problems or safe decisions need their own evidence. The more consequential the error, the less an average score can replace checking exceptions. Better work is a defensible conclusion only when the quality test matches what the work is for.

Sources: Experimental evidence on the productivity effects of generative artificial intelligence; Generative AI at Work

When one study really does measure productivity and quality

AI can improve both speed and judged output quality in a tested setting. The randomized professional-writing experiment in “Experimental evidence on the productivity effects of generative artificial intelligence” is a clear example: it measured time and quality for the same assigned work, rather than treating faster completion as proof that work was better. It supports joint task-level gains under those conditions, not a claim that every writing workflow or job will improve.

The preregistered online experiment enrolled 453 college-educated professionals. Participants received occupation-specific, incentivized midlevel writing tasks, and half were randomly given ChatGPT access. The published abstract reports average time taken fell by 40% and output quality rose by 18%. Random assignment strengthens the comparison of access versus no access in this setting. The two results answer distinct questions: how long the task took and how the submitted text was judged.

That second measure matters. If evaluators recorded only time, a quick but unusable draft could look like a gain. Here, output quality was also rated, so the result was not simply “less time, therefore better work.” For these participants and tasks, access improved measured speed and rated writing quality. This is stronger evidence for that narrow claim than a speed-only measure. When reading another productivity result, look for a task-relevant quality outcome instead of assuming that volume carries quality along with it.

The boundary is as important as the result. This was a short online task experiment with college-educated professionals, an early generation of ChatGPT, and writing assignments. The quality measure was a rating of submitted output, not whether a client later used it successfully, a decision based on it was sound, or corrections emerged downstream. One session also cannot show durable writing skill, lasting gains with repeated use, organization-wide value, or all workplace review costs. Those outcomes require separate evidence.

The distinction applies to ordinary work. A faster first draft for a report may help, but the quality question is whether it serves the report’s purpose and survives review. A shorter support reply may read clearly while failing to resolve the issue. These are illustrations, not findings from the experiment. In a low-consequence task with a stable acceptance test, a rating may closely track usability. Where errors surface later or affect others, a task rating is only one part of the evidence.

Neither extreme follows. Productivity evidence is meaningful when it measures a real task, and this experiment shows speed and judged quality can improve together. But it does not guarantee results in another workflow or establish downstream value, lasting capability, or job outcomes. Read it as an answer to a bounded question: what happened to time and rated writing quality on these tested tasks when participants received ChatGPT? To apply it at work, compare the study’s task and quality standard with the work you need done.

Sources: Experimental evidence on the productivity effects of generative artificial intelligence; Experimental evidence on the productivity effects of generative artificial intelligence

Why task boundaries can reverse the quality result

AI's contribution can change between neighboring tasks because apparent human difficulty does not reliably predict whether model output will help. In a complex workflow, a tool may improve one component and weaken another, even when both seem similarly demanding. If a worker treats output as dependable without recognizing that boundary, the combined result can be worse. “Navigating the Jagged Technological Frontier,” a 2026 peer-reviewed study in Organization Science, tested this idea with 758 consultants at Boston Consulting Group (BCG). In a preregistered randomized experiment, consultants completed realistic management-consulting tasks with no AI access, GPT-4 access, or GPT-4 access plus a prompt-engineering overview. On 18 tasks classified as inside the tested frontier, participants with AI completed 12.2% more tasks, worked 25.1% faster on average, and produced solutions of significantly higher quality than the control group. On a complex managerial task selected as outside that frontier, participants using AI were 19% less likely to produce correct solutions. That contrast is a reason to split a job into task components before drawing conclusions. Consider a consulting workflow with a request to synthesize supplied material and a separate request to validate a source-sensitive claim. The first asks for an organized account of available information; the second depends on whether each claim is supported by the right source and interpreted correctly. A strong draft on the first does not establish dependable performance on the second. This example was not tested in the study. The experiment also cautions against treating prompt skill as the whole explanation. Participants were assigned to a condition with a prompt-engineering overview as well as one without it, but performance varied with task fit. The authors report that consultants sometimes over-relied on AI when closer supervision was needed. A person may therefore need enough subject knowledge to notice a plausible but unsupported answer; asking for a polished explanation is not itself verification. Check output against evidence and task requirements, rather than judge it by fluency alone. The limits matter. This was a controlled experiment with BCG consultants, tasks designed to resemble consulting work, and GPT-4 as available in June 2023. It demonstrates a mechanism in that setting, not a current capability map, a forecast for other occupations, or organization-wide outcomes. Teams deciding where to use AI should test workflow components separately, with success criteria set before comparing assisted and unassisted work. Keep human review close where errors are difficult to spot, claims must be traced to sources, or a wrong answer has meaningful consequences. A task-level productivity gain is useful evidence, but a task-specific quality check is needed to show whether the gain was good work.

Sources: Navigating the Jagged Technological Frontier

What field evidence adds—and still leaves out

Field evidence adds a layer: it observes AI inside an operating workflow, where people, queues and workplace rules shape what counts as an outcome. But “real world” does not make a result universal. A field study still measures a particular task, workforce, tool and set of outcomes. Its value depends on how closely those conditions match the work a reader needs to assess. The Quarterly Journal of Economics study “Generative AI at Work” examined the staggered introduction of a conversational assistant among 5,172 customer-support agents at one firm. The publisher abstract reports a 15% average increase in issues resolved per hour. It also reports a split that an overall average can hide: less experienced and lower-skilled agents improved both speed and output quality, while the most experienced and highest-skilled agents saw small speed gains and small quality declines. The study further reports changes in customer interactions, including greater politeness and fewer requests to speak with a manager. This is stronger than a simple count of drafts because it includes a defined resolution measure and several quality or experience indicators. Still, the study concerns one company’s support workflow and a specific conversational assistant. Resolutions per hour do not, by themselves, establish whether the same customers needed repeat help later, whether every type of issue was handled well, or whether the gains transfer to another service operation. Those questions require their own measures and settings. A different kind of field evidence comes from “The Effects of Generative AI on High-Skilled Work: Evidence from Three Field Experiments with Software Developers,” published in Management Science. Across randomized trials at Microsoft, Accenture and a Fortune 100 company, 4,867 developers were assigned access or no access to a coding assistant during ordinary company work. The article reports that the combined estimate was 26.08% more completed tasks for developers with access, while noting that individual experiment results were noisy and varied. That outcome is meaningful: completed tasks are closer to work delivered than suggestions generated. But the abstract does not make completed-task counts equivalent to comprehensive software quality. Without corresponding measures of defects, maintainability, review effort or downstream incidents, it cannot settle whether all completed work was better work. These results are not a contradiction. A support issue resolved per hour and a developer task completed are different units, populations and workflows. Each can support a bounded claim about measured productivity; neither alone captures the whole value chain. For a reader comparing evidence with their own role, the useful question is whether the study’s acceptance rules, user mix and downstream checks resemble theirs. A workplace result travels farther when those features match and when quality is measured at the point where someone depends on the output. If they do not, the finding may still suggest what to test, but it is not yet evidence that the reader’s work became better.

Sources: Generative AI at Work; The Effects of Generative AI on High-Skilled Work: Evidence from Three Field Experiments with Software Developers

How to read a striking result that later needs qualification

A study remains useful when its conclusion stays attached to the people, tasks, tools and dates it actually tested. A later update can change how far that conclusion travels, or whether it describes current practice. Keep a historical estimate in its tested setting rather than treating it as a general claim about what workers should expect now. METR’s “Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity” reports a randomized trial involving 16 experienced contributors to large open-source repositories. Across 246 issues, developers were assigned tasks in which AI use was allowed or disallowed; the AI condition primarily involved Cursor Pro with Claude 3.5 or 3.7 Sonnet. METR reports that developers took 19% longer when AI was allowed,. The result concerns this group, familiar repositories, early-2025 tools and the study’s task and timing rules. It does not show that AI slows most developers or other occupations; METR says these claims are not established. The February 2026 post “We are Changing our Developer Productivity Experiment Design” does not replace it with a universal estimate. METR says the later study’s signal is unreliable, in part because developers who did not want to work without AI were less likely to participate. The researchers also identify a lower participation payment, from $150 to $50 an hour, as a possible source of selection, and say time measurements were unreliable for some developers running multiple agents concurrently. METR reports raw estimates pointing toward speedup in some groups, but explicitly warns that selection makes them weak evidence for its size. They are not a settled reversal of the earlier slowdown. This update reveals a measurement problem that becomes more important as use spreads: the act of adopting a tool can change the comparison group. If people expecting the largest benefit are less willing to enter a condition without AI, a randomized assignment among volunteers may no longer represent the broader group of users or tasks. If developers also choose which tasks to submit based on whether AI is available, the task mix can shift. And if concurrent agents make personal work time hard to record, the denominator becomes less clear. Randomization within a study cannot repair nonparticipation before enrollment or a time measure that no longer captures effort consistently. When reading a striking result, identify its population, task, tool version and period; inspect how output and time were measured, and check later design notes. METR’s early trial is evidence about its 2025 setting; the later post documents how adoption complicated measurement. Neither finding should be turned into a current forecast for an individual’s productivity, much less a conclusion about job loss. Give more weight to studies that define the task mix, capture work and review time, report participation and missing data, and measure outcomes beyond speed. Treat the slowdown as a bounded historical finding and later raw speedups as inconclusive, rather than choosing whichever headline confirms a prior belief.

Sources: Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity; We are Changing our Developer Productivity Experiment Design

Why faster individual work does not settle the workplace result

An AI-assisted task can take less time and still leave the workplace result unchanged. Between an individual task and an organization-wide outcome, someone must use the system, accept its output, fit it into a workflow, and decide what to do with time saved. A faster first draft is evidence about drafting under measured conditions; it does not show that a team completed more customer work, improved service, or changed staffing. The International Labour Organization's June 2026 review brings together experiments, firm-level data, platform studies and worker and firm surveys from Australia, Denmark, Germany, Korea, Kuwait, the United Kingdom and the United States. Its summary says productivity gains are real, though often unverified and uneven. It also reports that worker-reported time savings of a few per cent of working hours had not yet translated into higher measured output, earnings or employment in the evidence reviewed. This is a distinction, not proof that saved time never matters: reported minutes, verified individual output and labor-market outcomes are separate measures. The review covers varied studies and settings, so it does not establish what happened in every workplace or what will happen next. The gap between saved minutes and wider results has several possible explanations. A worker may spend the time checking a draft, correcting errors, or coordinating a handoff; if those tasks were not included in timing, gross speed can overstate net savings. A manager may use released capacity for more volume, other duties, or no immediate change. A workflow may need adaptation before accepted outputs move smoothly to the next person. These are mechanisms to investigate, not findings about a particular employer. Measure the task path, including review and rework, then separately observe the outcome the next person or customer needs. Do not count a generated item as completed work when acceptance still depends on review. The OECD review, “The effects of generative AI on productivity, innovation and entrepreneurship,” says effectiveness depends on the user's experience and the task, and identifies long-term business effects and workers' understanding of AI limits as research gaps. That cautions against treating a task result as a firm result. A capability may be available without being used consistently, and observed use does not show that an organization has redesigned roles or processes around it. Output gains alone do not show how benefits or burdens are distributed among workers. For a worker interpreting an experiment, keep the claim proportional to its measure. Say a defined task became faster under tested conditions if that is what was measured. To claim better workplace performance, look for accepted output, review cost and a relevant downstream result; to claim changed employment or demand, require evidence at that level. This does not diminish a genuine productivity gain. It shows what additional evidence connects it to the work organizations ultimately care about.

Sources: The impact of GenAI on jobs, productivity and work organization: a review of the empirical evidence; The effects of generative AI on productivity, innovation and entrepreneurship

Better work includes worker cost and task redesign

Whether work is better can include how it is organized and experienced, but those dimensions should be named and measured rather than smuggled into an output-per-hour statistic. A tool may shorten one step and still leave open who checks its result, handles exceptions, answers for mistakes, and absorbs work that moves elsewhere. Those are questions about task design and worker cost, not just speed. The OECD report The Impact of AI on the Workplace: Evidence from OECD Case Studies of AI Implementation draws on nearly 100 qualitative workplace case studies in manufacturing and finance across eight countries. Some cases describe automation reducing tedious work or improving physical safety; others record higher performance targets, greater task complexity, work intensity, and stress. Fewer minutes on one task can therefore coexist with a faster pace or new responsibilities. These selected cases identify mechanisms, not a representative estimate of how often outcomes occur. They also concern AI implementations before today’s generative AI tools became widespread, so they are not a current causal estimate for generative AI. Task redesign can move effort rather than remove it. Consider a workflow that uses a system to draft a routine summary. Its user may now check factual details, correct exceptions, and explain the final decision to a colleague. If review is brief and the summary sound, tedious effort may fall while human judgment remains at the handoff. If the reviewer must reconstruct context or check every claim, the drafting-stage saving may conceal added downstream work. This is an illustration of what to measure, not a finding about a particular workplace. The distinction between automation and assistance matters because it changes who attends to surrounding tasks. When a step is automated, ask whether monitoring inputs, checking outputs, resolving unusual cases, or coordinating with the next person expands. When a tool assists a worker, ask whether it supports judgment or mainly raises expected volume. The OECD case studies describe complementary changes as well as cases where human work around an automated task increased. Because the cases are selected, they support asking about task boundaries, not predicting how a particular role will change. Worker experience needs its own evidence. The OECD paper Job Quality, Health and Productivity: An evidence-based framework for analysis describes associations between working-environment quality, health, work-life fit, skill use, and productivity. It is a general framework, not an AI intervention study, and cannot show that a specific system improves health or productivity. It does explain why those outcomes should not be collapsed into one number. Ask whose time was saved, whose checking burden grew, whether targets changed, and whether the person accountable retains the knowledge and authority to challenge the result. A credible account of better work makes those costs visible alongside accepted output.

Sources: The Impact of AI on the Workplace: Evidence from OECD Case Studies of AI Implementation; Job Quality, Health and Productivity: An evidence-based framework for analysis

How can I test whether AI made one task better?

Compare one bounded task with and without AI using a shared quality floor, count worker time through verification and rework, and inspect a downstream outcome. This practical synthesis is not a validated instrument. It shows what changed in the task, not whether a role will disappear. Define the unit of work and accepted output. A “case summary,” for example, needs an agreed purpose: capture the issue, relevant history, next action and any required escalation. Set the same minimum standard for assisted and unassisted work, including serious errors that make an output unacceptable even when it reads well. The standard should fit the use; an internal draft and a customer-facing answer may tolerate different defects. Record a baseline: items completed, items passing review, and producer and reviewer time. Compare a similar set under the assisted workflow. Keep the task definition, input quality, deadline and acceptance rule stable where possible. If cases differ, match them by type or difficulty. A small trial may guide what to examine without settling the result. Count the whole handoff: prompting, fact checking, correction, exceptions and explanation to the next person. Track rejected items and rework as well as accepted volume. Otherwise, a quicker draft can appear to save time while shifting uncounted work to a reviewer. In “Experimental Evidence on the Productivity Effects of Generative Artificial Intelligence,” 453 college-educated professionals were randomly assigned ChatGPT access for occupation-specific writing tasks. Researchers measured time and rater-assessed quality, finding both improved in that setting. Its ratings do not establish downstream value in another workflow. Choose one downstream check that matters to the recipient. For a support summary, it might be whether the next agent can act without reopening the history; for a report draft, whether a reviewer catches a material factual correction before use. These are possible measures, not reported study outcomes. Agree on the check before the trial. “Generative AI at Work,” a customer-support field study of 5,172 agents, examined issues resolved per hour and interaction quality, with results varying by worker experience. One employer’s operation cannot represent every workplace. Where practical, have reviewers apply the same criteria without knowing which version used AI. Otherwise, use a written rubric and record who rated each item. Note missing data, unusual cases and disagreements. The International Labour Organization’s “The impact of GenAI on jobs, productivity and work organization” review finds that worker-reported time savings have not yet translated into higher measured output, earnings or employment in the evidence it reviews. This supports separating task time from later outcomes; it does not predict what a particular team will find. Report the dimensions plainly: faster, better on the chosen quality measure, both, mixed, or not yet assessable. State why and name the uncertainty, such as too few comparable cases or untracked reviewer time. If speed improves while rework or downstream corrections rise, the result is mixed and the handoff needs attention. If speed and task-relevant quality improve while review burden stays stable, say this task improved under the tested conditions. The evidence answers a bounded work question, not an individual displacement probability.

Sources: Experimental evidence on the productivity effects of generative artificial intelligence; Generative AI at Work; The impact of GenAI on jobs, productivity and work organization: a review of the empirical evidence

Use the result to choose a proportionate next step

Start with one recurring task and agree with its recipient or reviewer on what “better” means. Define an acceptable result, total time spent producing and checking it, and one downstream sign that matters. Compare the same kind of work; do not turn a local test into a forecast about a role.

Read the result in parts. If accepted output improves and total effort falls without a quality decline, record that the tested task became faster under those conditions. The International Labour Organization’s review of empirical evidence on generative AI and work describes a gap between reported time savings and measured output, earnings, or employment. If speed rises while quality or reviewer burden worsens, redesign the handoff or retain human checking where it catches consequential errors. If the result is mixed, narrow the next comparison to the step or error type still unclear.

There is a fair exception: for stable, low-consequence work with a reliable acceptance test, throughput may closely track quality. Otherwise, keep the checks separate. A reviewer conversation can establish what to measure within your time, income, health, and family constraints. If you need to understand which tasks show change pressure, the free task checker offers task-level signals and first actions. Those signals are not a validated probability of displacement; a productivity result alone does not establish job loss.

Sources: The impact of GenAI on jobs, productivity and work organization: a review of the empirical evidence; Navigating the Jagged Technological Frontier

Questions readers ask