Do AI Productivity Gains Also Improve Work Quality?

Can a faster task also be better?

Yes. A randomized experiment found that professional writers completed short assignments faster and produced work that evaluators rated more highly with ChatGPT. In the final Science report, 453 college-educated professionals completed occupation-specific writing tasks; completion time fell by 40%, while assessed output quality rose by 18%. This is evidence that productivity and measured quality can improve together on a bounded task. It is not evidence that every faster workflow produces better work. The distinction matters because the study measured the submitted writing against an evaluation rubric, not its later performance in a workplace. A stronger rating supports a claim about those outputs under those conditions. It does not, by itself, show that the text was factually correct for a particular client, reduced later revisions, improved a business decision, or remained useful after being acted on. Those are separate outcomes and would need to be measured separately. So a worker need not assume that speed requires a quality tradeoff, but should not treat a fluent, quickly completed draft as proof of improvement either. The relevant question is what the rating captures and whether that dimension matters in the actual task. If quality means accurate figures, a sound recommendation, or fewer corrections after handoff, a study rating on a short writing assignment cannot answer that local question. The result establishes possibility, not a general quality premium: time saved is productivity evidence; quality must be checked against the standard the work is meant to meet.

Sources: Experimental evidence on the productivity effects of generative artificial intelligence | Science

What exactly counts as work quality?

Quality is not one universal score. It can mean whether a statement is factually correct, whether an answer solves the client's problem, whether a response resolves a service issue, whether code passes tests, or whether later users can maintain it without costly repair. A study can show improvement on its chosen measure; it cannot silently establish every other meaning of better work. That distinction is central when asking whether AI productivity gains also improve quality.

In the randomized professional-writing experiment reported in “Experimental evidence on the productivity effects of generative artificial intelligence,” evaluators rated short, occupation-specific writing outputs. The reported quality result therefore concerns how those submitted pieces performed under the study's rating process, alongside completion time. It is evidence that faster work and higher assessed quality can coincide in that bounded setting. It does not by itself tell a reader whether a particular memo is factually complete, whether a client acted on it successfully, or whether corrections surfaced after delivery.

“Navigating the Jagged Technological Frontier” uses human-graded consulting answers and correctness on selected tasks. Correctness on a constructed managerial problem is not the same as usefulness across a live client engagement. The “Generative AI at Work” study follows a customer-support operation and reports resolutions per hour as its main productivity measure, while also examining service-quality indicators such as resolution and customer satisfaction. That setting is more operational than a one-off writing task, yet resolution and satisfaction capture only selected parts of service quality. It cannot stand in for every customer's needs or all downstream consequences.

A code experiment makes the dimensions especially visible. GitHub's “Does GitHub Copilot improve code quality? Here’s what the data says” describes a randomized task completed by experienced developers, with functionality checked using ten unit tests and anonymized submissions reviewed for readability and other code attributes. The company reports differences across functionality, readability, reliability, maintainability, concision, and approval. These checks are not one quality score: passing tests does not guarantee that code will be easy to change in a production system, while a readability review cannot establish that every function behaves correctly. The study is also vendor-led and uses a constructed web-server assignment, so its rubric illustrates multidimensional assessment, not proof about long-lived software projects.

The practical question is therefore not simply, “Did quality improve?” but, “Which quality dimension improved, for whom, and at what point in the workflow?” For a customer update, correctness and whether it answers the customer's actual question may matter more than polish. For a code change, tests and review can catch some failures, while maintainability may only become clear later. For a research summary, a clean structure is not a substitute for verified claims. Match the evidence to the consequence that matters in your task. More lines, tickets, or drafts show volume; unless the relevant quality outcome was checked too, they do not establish fewer errors or more useful work.

Sources: Experimental evidence on the productivity effects of generative artificial intelligence | Science; Navigating the Jagged Technological Frontier | Organization Science; Generative AI at Work | The Quarterly Journal of Economics; Does GitHub Copilot improve code quality? Here’s what the data says | GitHub Research

Why can one workflow produce gains and losses?

The same AI assistant can improve one part of a knowledge worker’s job and make another less reliable. The useful unit of analysis is the task-tool-workflow combination, not the occupation as a whole. A task may fit the system’s demonstrated strengths when its inputs are clear, its output is bounded, and a person can check the result. A neighboring task may require judgment the system handles poorly, or make a plausible error difficult to notice. This helps explain why productivity and quality findings can point in opposite directions without either being meaningless. In “Navigating the Jagged Technological Frontier,” researchers ran a preregistered experiment with 758 consultants at Boston Consulting Group. Participants worked on firm-developed tasks pretested as within or beyond the capabilities of the GPT-4-era system. On within-frontier tasks, AI access improved performance as judged by human graders. On a selected task beyond that boundary, AI users were less likely to give a correct answer. The study compared task performance across a capability boundary; it did not classify whole consulting jobs as suitable or unsuitable for AI. The contrast matters because users cannot always see the boundary in advance. A fluent answer can look finished even when the task calls for reasoning the tool does not perform reliably. If a worker treats fluency as evidence of correctness, assistance can encourage reliance where independent checking matters. The experiment supports caution about that pattern, but does not establish that every error came from over-reliance or that every user behaved the same way. It is evidence about selected tasks under particular experimental conditions, not a forecast of routine performance in every firm. “Within the frontier” should not become a permanent label for a job duty. The experiment used specific tasks, a particular system generation, and a defined setting. Tools change, as do prompts, source material, review steps, and the cost of an error. A task may be manageable when a worker can compare an answer with a clear reference, yet riskier when it guides a consequential decision and no one can verify it cheaply. Capability is only part of fit; workflow and oversight shape whether it produces usable work. For a practical trial, separate repeatable steps from final judgment. Drafting options or organizing supplied material can be tested on its own; approving a recommendation should be evaluated separately when its evidence, assumptions, or consequences are harder to check. Increase scrutiny with the stakes and uncertainty about reliability. A good result on ideation or an early draft is a reason to test that step further, not proof that AI can safely make the final call. This task-level approach preserves useful assistance while keeping responsibility for consequential judgments visible.

Sources: Navigating the Jagged Technological Frontier | Organization Science

What changes when quality means a live service outcome?

A live customer-support operation makes the quality question more practical, but one average productivity figure is not a complete quality verdict. *Generative AI at Work* studied a staggered rollout among 5,172 agents at one large software firm. Its assistant suggested replies during customer chats. Access was associated with 15% more issues resolved per hour on average. The researchers also examined the components separately: time per chat, chats handled per hour, the share of conversations resolved, and customer satisfaction from post-chat surveys. They report a small average rise in resolution rate from a high baseline, and no economically meaningful average change in surveyed satisfaction. So the operation became more productive without an average decline on these indicators; the throughput gain alone does not show that every interaction improved. The distribution across workers complicates the average. Less-skilled and newer agents generally saw larger gains in productivity and measured quality. For the highest-skilled group, average handling time did not improve; chats per hour rose, while resolution rates and customer-satisfaction scores fell slightly. Newer workers’ gains appeared in quality measures, while more experienced groups had small declines in customer satisfaction. These are subgroup results from this study, not a rule that experienced workers are harmed by assistance. The authors suggest recommendations may help newer agents while distracting some already high-performing agents, but that explanation is an interpretation of this setting, not a universal mechanism. The results do show why a team-wide mean can hide differences in how the same workflow affects workers. Operational evidence answers a different question from a short exercise graded by evaluators. Real chats involve customer problems, repeated use, and measures such as resolution and satisfaction, making this setting relevant to service design. Yet it remains one company’s support operation, with a particular assistant, customer base, training, and division of responsibility. Resolution reflects whether a conversation solves an issue under the firm’s process; satisfaction comes from sampled post-chat surveys. Neither captures every dimension a customer or employer might care about, such as safe handling of a rare edge case or later rework. The firm also lacked call-quality measures for every agent, limiting some comparisons. This is closer to workplace performance than a one-off writing score, but not a universal audit of service quality. The authors also report evidence consistent with worker learning and improved English fluency, particularly for international agents. Their experience-curve analysis found that agents with access moved more quickly on measured performance during the observed period. But the paper notes that its design cannot distinguish learning underlying skills from following useful recommendations. A worker may produce a stronger response with assistance without showing what they can do unaided later. Teams evaluating a rollout should therefore inspect resolution, customer feedback, correction or escalation patterns, and workload by task and experience group, not just aggregate issues per hour. An average gain is a reason to investigate where the workflow helps; it does not establish that every colleague or interaction improved.

Sources: Generative AI at Work | The Quarterly Journal of Economics

Does more completed work mean better work?

No conclusion about quality follows from a task-count or time-saving result unless quality was also measured. More completed work may mean more acceptable work, but it may also mean workers finished routine steps faster or produced more work needing review. Productivity evidence can show a change in time or volume without showing whether results became more accurate or useful. The six-month randomized field experiment reported in “Shifting Work Patterns with Generative AI” followed 7,137 knowledge workers across 66 firms. Workers were randomly selected for access to a generative AI tool in their email, meeting, and writing applications. In the second half of the experiment, 80% of treated workers who used the tool spent two fewer hours per week on email and reduced work outside regular hours. The researchers did not detect changes in the quantity or composition of tasks resulting from individual-level access. This is evidence about time use and measured task patterns, not a direct measure of email, meeting, or writing quality. “Did not detect” also does not mean that no individual or firm changed how work was organized. Another productivity result appears in “The Effects of Generative AI on High-Skilled Work: Evidence from Three Field Experiments with Software Developers.” The randomized trials at Microsoft, Accenture, and an unnamed Fortune 100 company covered 4,867 developers. Across the combined data, access to an AI coding assistant was associated with a reported 26.08% increase in completed tasks, although the authors note that results were noisy and varied across the three experiments. The published abstract measures completed-task counts; it does not report a quality assessment of those tasks. A completed task is not thereby proven correct, secure, maintainable, or valuable in the wider software system. These studies therefore answer a different question from experiments that ask evaluators to rate a document or test whether code meets a defined rubric. One study reports saved email time and less after-hours work; the other reports greater task completion. Neither establishes that work quality improved. The studies also do not establish what happened to every saved hour. Shorter days, more output, checking, or other responsibilities are possibilities, not reported results. For a worker deciding whether an AI-assisted process is better, keep the measures separate. Record end-to-end time and volume, then check the quality criteria that matter in the task: for example, whether a draft is accurate and usable, or whether a code change passes relevant tests and survives review. Include correction and rework rather than counting only the first output. If quality was not measured, describe the finding as a time or throughput gain and leave the quality question open. Faster work may create room for improvement; the evidence must show that improvement before speed becomes a quality claim.

Sources: Shifting Work Patterns with Generative AI | American Economic Review: Insights; The Effects of Generative AI on High-Skilled Work: Evidence from Three Field Experiments with Software Developers

Where can a speed gain turn into a quality gain?

A speed gain can become a quality gain when the saved effort is put into checking, refining, or correcting the work before it reaches someone who depends on it. The sequence matters: assistance handles a bounded step; that step takes less effort or surfaces a defect sooner; a worker uses the remaining capacity to review and improve the result; and the correction happens before handoff. If the time saved is instead absorbed by a higher output target, or if review is skipped, productivity may rise without any improvement in the finished work. The evidence supports this as a plausible workflow mechanism, not an automatic consequence of using AI. A randomized GitHub study illustrates one version of the chain in software. The company recruited developers with at least five years of experience; 202 valid submissions came from a controlled exercise to build API endpoints for a web server, with one group assigned Copilot and another told not to use AI tools. Functionality was checked against ten unit tests, and developers who were blind to which condition produced the code reviewed submissions. Copilot-assigned developers were more likely to pass all ten tests, and the blind reviewers reported fewer readability errors and modestly higher ratings across several code dimensions. The task gave developers feedback they could act on: tests exposed whether functions worked against specified cases, while review identified issues in how the code was written. It was a vendor-led trial, not a test of long-lived production systems,. A different mechanism appears in the Organisation for Economic Co-operation and Development’s workplace case studies. The 2023 report draws on nearly 100 qualitative cases of AI technologies in manufacturing and finance across eight countries, including applications that automated tedious tasks such as quality-assurance inspection. It describes systems that could improve detection or correction and free time for other tasks. This evidence predates today’s generative tools and covers varied technologies and sectors, so it cannot quantify a current generative-AI quality effect for knowledge workers. It illustrates why early defect detection can make correction easier, while automation may leave capacity for work requiring attention. It also reports a countervailing possibility: higher output targets can intensify work instead of allowing careful review. The distinction is practical for a team introducing assistance into drafting, analysis, or coding. Quality can improve if a person has enough time and a clear standard to check important claims, test behavior, repair weak reasoning, or tailor the output to its intended use. Conversely, a polished result can carry an unchecked factual error, and a test suite can miss requirements it never represents. So, if quality is the goal, make review part of the workflow rather than assuming saved minutes will turn into better work. Preserve time for a check, and track corrections or rework alongside task time. The coding and workplace cases show ways productivity can support better results; they do not show that every saved hour is reinvested well. Without that deliberate step, an organization may simply get faster completion or more output.

Sources: The impact of AI on the workplace: Evidence from OECD case studies of AI implementation; Does GitHub Copilot improve code quality? Here’s what the data says | GitHub Research; Experimental evidence on the productivity effects of generative artificial intelligence | Science

Does assisted quality carry over to unassisted judgment?

Immediate output quality and retained expertise are different outcomes. In the three-month randomized field experiment “Does AI Assistance Enhance or Erode Expertise? Evidence from a Three-Month Field Experiment in Patent Drafting,” 133 practicing patent lawyers at 11 U.S. intellectual-property firms received access to a custom drafting assistant or served as controls. Blinded expert patent attorneys scored their work. On benchmark drafting tasks, scores for the assisted group were higher at 10 days and 90 days: 0.34 and 0.38 standard deviations, respectively. These are ratings of specified tasks, not a general measure of legal service quality. The study also tested performance without AI. After three months, participants redlined an existing patent application, a task requiring professional judgment. The treated group scored 0.32 standard deviations above controls on average. The advantage was concentrated among senior lawyers, whose scores were 0.45 standard deviations higher. Junior lawyers showed no average gain. Their scores spread in both directions: fewer were mediocre, but more were poor as well as good. The average does not describe every participant; results differed by experience level. This trial complicates the idea that assistance must either teach everyone or make everyone dependent. Judged quality rose during assisted drafting, and the treated group later performed better on average without the tool. Yet that carryover appeared among senior lawyers, not as an average gain for juniors. One interpretation is that established expertise may help people use assistance productively, while less experienced workers may see more variable effects. The study authors suggest foundational expertise may be a prerequisite for durable skill gains. This is a possible explanation, not proof of the mechanism or a rule about all employees. The evidence is relevant to carryover because it uses random assignment, blinded scoring, a three-month period, and a later task completed without AI. But 133 lawyers, one specialist practice, a custom assistant, and one unassisted assessment cannot establish long-term retention or generalize to other professions. A single task cannot show how expertise develops across a career. The result supports the possibility that assisted work and later unassisted judgment can improve together in a defined context; it does not show that AI reliably teaches or preserves skill. Where expertise matters, a local tool trial can track two outcomes separately: assisted deliverables against agreed criteria, and whether the worker can explain, check, or perform a representative step without assistance. The second measure is useful when independent judgment matters or errors carry consequences; it need not become a ritual for every routine task. Both measures help distinguish stronger work from delegated effort, with evaluation sized to the task.

Sources: Does AI Assistance Enhance or Erode Expertise? Evidence from a Three-Month Field Experiment in Patent Drafting | Google Research

What does the evidence justify across knowledge work?

Across knowledge work, AI assistance can improve productivity and measured quality together when the task fits the system and outcomes are checked, but there is no general quality premium. These studies test different tasks, workers, tools, time horizons, and definitions of quality. Read them as complementary cases, not estimates to average into one verdict.

Short randomized experiments can isolate task-level effects more clearly than workplace comparisons. In “Experimental evidence on the productivity effects of generative artificial intelligence,” 453 college-educated professionals completed bounded writing assignments; the Science report found average completion time fell 40% and evaluator-rated quality rose 18%. This shows speed and rated quality improving together on those assignments, not necessarily in client work where accuracy, revision, and downstream usefulness may matter beyond the rubric. “Navigating the Jagged Technological Frontier” tested consultants on selected tasks. Gains within the tested frontier contrasted with lower correctness on a task beyond it, a counterexample to assuming one successful use transfers across a profession.

Workplace evidence adds operational context but remains setting-specific. “Generative AI at Work” followed a staggered assistant rollout among 5,172 customer-support agents and reported 15% more issues resolved per hour on average. Quality results varied by experience: less experienced agents improved on speed and quality measures, while the most experienced group had small declines in some quality measures. Throughput alone therefore gives an incomplete account of service quality. The six-month randomized “Shifting Work Patterns with Generative AI,” across 66 firms and 7,137 knowledge workers, found email-time savings among users but no detected task-quantity or composition change from individual access. Its abstract does not report output quality, so it cannot show that saved hours improved work products.

Other evidence tests dimensions that time misses, with limits. “Does GitHub Copilot improve code quality? Here’s what the data says” describes a vendor-led randomized coding study using unit tests and blind developer ratings across several code attributes. This is more informative than code volume alone, but a constructed task does not establish maintainability in production systems. The three-month patent-law experiment “Does AI Assistance Enhance or Erode Expertise?” adds an unassisted measure: among 133 lawyers at 11 U.S. intellectual-property firms, benchmark drafting quality improved with assistance, and later unassisted redlining performance was higher on average, concentrated among senior lawyers. Its specialist sample and short duration leave longer-term skill effects unresolved.

The conclusion is yes in some measured settings, uncertain in many real workflows, and never established locally by speed alone. Confidence would grow with replicated, longer-run, multi-occupation studies measuring accepted quality, corrections, downstream rework, time, and output. For now, treat each result as evidence about its particular task and measure. Better performance on a bounded assignment can justify a careful local test against criteria agreed before use, and the consequences that matter in the reader’s actual work context; it cannot certify a whole workflow, prove lasting expertise gains, or establish broad adoption or displacement.

Sources: Experimental evidence on the productivity effects of generative artificial intelligence | Science; Navigating the Jagged Technological Frontier | Organization Science; Generative AI at Work | The Quarterly Journal of Economics; Shifting Work Patterns with Generative AI | American Economic Review: Insights; Does AI Assistance Enhance or Erode Expertise? Evidence from a Three-Month Field Experiment in Patent Drafting | Google Research

How can I check quality in my own task?

Run a small, reversible comparison on one recurring, low-risk task. Define acceptable work first, compare assisted and usual outputs on similar cases, and count time spent checking, correcting, and handling later rework. The aim is to learn whether this task is faster at the same or better standard in your setting, not to prove AI improves your whole role. “Navigating the Jagged Technological Frontier” is a reason to keep the test narrow: results differed between tasks within and beyond the tested capability boundary. “Generative AI at Work” also found variation by agent experience and quality measure. Neither study supplies a universal threshold for your work. Choose a task with a clear finish line and an inspectable result. A first draft of a routine internal update may be suitable if it contains no sensitive information and a person remains responsible for what is sent. An open-ended task such as “make this account plan better” is harder to compare: workers may define better differently, and the work may change during evaluation. Write down criteria for the task. For an update, these might include whether each material fact matches its source, the intended reader can identify the decision or next step, and the tone fits the relationship. Use criteria that reflect the consequences; a clean sentence is not a substitute for a correct claim. Compare similar pieces of work. Pair cases with similar purpose, length, and difficulty, or alternate between usual and assisted processes where practical. Record end-to-end time, including prompting, checking, edits, and corrections after handoff. Note what changed: factual errors, missing context, unclear recommendations, or reviewer-requested rework. If work affects a client, money, safety, or a consequential decision, ask a qualified person to check relevant claims before use. Do not count unchecked output as acceptable just because it reads smoothly. The OECD workplace case report describes settings where AI supported quality aims, but does not establish that saved effort automatically becomes better output. That depends on how the workflow uses the time and whether defects are caught. Before starting, decide what would make you continue, narrow the tool to one step, or stop. There is no universal pass mark for every task. Continue only if the comparison meets your criteria and review costs do not erase the practical gain. Narrow or stop if important errors remain, an answer cannot be checked, or correction work outweighs time saved. If results are mixed, keep the useful step and redesign the handoff or review instead of declaring the whole workflow a success or failure. Repeat the comparison when the task, tool, or stakes change materially. When cases differ, record that too: a comparison cannot isolate the tool's contribution if one set of tasks was simply harder. This checks task quality and effort locally; it does not show broad employer adoption, predict job loss, or prove expertise is growing or declining.

Sources: Navigating the Jagged Technological Frontier | Organization Science; Generative AI at Work | The Quarterly Journal of Economics; The impact of AI on the workplace: Evidence from OECD case studies of AI implementation

What is the next move if AI has sped up my work?

If AI has made a recurring task faster, run a bounded quality check. Choose one low-risk task with a clear finish, such as preparing an internal summary. Before using assistance, write down what an acceptable result must contain and which errors matter. Compare assisted work with your usual process on similar cases. Record total time through review, corrections, and handoff; ask a qualified colleague to check consequential parts when accuracy matters. This is a local workflow test, not evidence that your occupation is safe, that your employer has adopted AI broadly, or that anyone will lose a job. “Navigating the Jagged Technological Frontier” found different results inside and outside the tested task boundary. It supports checking fit, not assuming every task behaves alike. Agree with your manager or collaborator on a review date and a practical decision: continue, narrow the use, or stop if errors and rework outweigh time saved. Ask: “Can we try this on one recurring, low-risk task, agree what quality means, and review it together next month?” To map which parts of your role face change pressure, the free task checker offers transparent task-level signals, not a validated probability of job loss. A career roadmap is a separate option for comparing paths against your constraints.

Sources: Navigating the Jagged Technological Frontier | Organization Science

Questions readers ask