Who Benefits Most From AI—and Do the Gains Last?

Short answer: the biggest immediate gain does not identify the biggest learner

People starting from a lower measured baseline or with less experience often gain more on a task when AI assistance is available. That is a finding about output under assistance, not a reliable answer to who learns most. After access ends, some studies find partial carryover, others find no independent advantage, and some find worse unaided performance. The result depends on the task, how the person uses the tool, and what the follow-up measures. For recurring work, ask what the person and tool produce together and what the person can do when the tool is absent. The randomized online study “Does generative AI narrow education-based productivity gaps? Evidence from a randomized experiment” assigned 1,174 adults aged 25–45 to workplace-style problem-solving tasks. Its abstract reports that AI improved performance across education groups and narrowed the measured gap from 0.548 standard deviations without assistance to 0.139 with it. In the unassisted follow-up, the gap re-emerged, although participants with less education retained part of their gain; the abstract also links improved follow-up with sustained effort. This shows partial carryover in one setting; education is not interchangeable with task performance, expertise or job tenure. The field study “Generative AI at Work” examined a deployed customer-support assistant. It reports a 15% average increase in issues resolved per hour and a 30% increase for less-skilled and less-experienced agents in that setting. But the study measured work with the system in use; it was not a randomized test of what agents could do after withdrawal. Faster assisted production cannot, on its own, establish retained independent skill. A different pattern appears in “Does AI Assistance Enhance or Erode Expertise? Evidence from a Three-Month Field Experiment in Patent Drafting.” In a randomized trial involving 133 patent lawyers at 11 firms, the accessible research summary reports that junior lawyers gained more in assisted drafting, while the average later unaided professional-judgment gain was concentrated among senior lawyers. The small, specialized sample cannot represent every workplace, but shows that the largest assisted gain need not mean the largest later independent gain. Taken together, these studies support a careful answer: lower-baseline or less-experienced participants can benefit substantially while assistance is present, yet that advantage does not reliably predict learning after it is removed. None gives an individual worker a probability of losing a job. For a recurring task, a polished deliverable shows what the combined workflow achieved; it does not tell you whether the task is now learned or whether your independent judgment improved. If both outcomes matter, look for evidence of both before calling assisted output durable skill.

Sources: Does generative AI narrow education-based productivity gaps? Evidence from a randomized experiment; Generative AI at Work; Does AI Assistance Enhance or Erode Expertise? Evidence from a Three-Month Field Experiment in Patent Drafting

Who gains most while the tool is available?

The largest immediate gains often go to people who start with a task-specific gap: less experience, lower measured performance, or less familiarity with the problem. Education, a score on one task, job tenure, and broad expertise are different measures. AI can raise current output most when it supplies examples, drafts, or procedural cues that a worker would otherwise need time or experience to find. It does not establish that every novice benefits more, or that the person with the largest assisted gain learns the most. In “Does generative AI narrow education-based productivity gaps? Evidence from a randomized experiment,” 1,174 adults aged 25–45 were randomly assigned to complete a workplace-style problem-solving task with or without AI, then did an unassisted module. Performance improved across education groups, with larger gains among those with less education. Without AI, higher-education participants led by 0.548 standard deviations; with AI, the gap was 0.139, closing about three-quarters of the initial difference on the assisted task. After access ended, participants did not perform worse overall, and lower-education participants retained part of their improvement, although a sizable gap returned. Education is not a direct measure of tenure or task skill. A deployed support setting shows a related pattern with a different outcome. “Generative AI at Work,” a study of 5,172 customer-support agents at one business-software firm, examined staggered access to an assistant suggesting replies during chats. The Quarterly Journal of Economics article reports a 15% average rise in issues resolved per hour. Less experienced and lower-skilled agents improved more in speed and quality; the most experienced and highest-skilled saw small speed gains and small quality declines. Newer agents with AI access also moved down the observed experience curve faster. This was a production workflow, not a randomized test with the tool removed. Faster learning while supported is suggestive, but does not cleanly measure independent performance after withdrawal; one occupation at one firm cannot stand in for all knowledge work. “Experimental Evidence on the Productivity Effects of Generative Artificial Intelligence” provides a third comparison. In this randomized online experiment, 444 college-educated professionals completed short, occupation-specific writing tasks, which blinded professionals rated. ChatGPT reduced time by about 37% relative to control and raised average grades by 0.45 standard deviations. Participants who scored lower on the first task gained more in grades, while higher scorers largely maintained grades and reduced time. The authors found that the tool mostly substituted for effort in this setup: many users submitted initial generated text with little editing. These brief tasks did not test retained writing skill. “Lower baseline” means a lower score on the initial task, not less education, tenure, or professional potential. Together, the studies show how assistance can narrow a measured output gap by making task knowledge easier to access. A suggestion may help a newer agent handle an unfamiliar issue, or a draft may help a writer get past a slow first pass. These are plausible explanations, not proof of one mechanism. So “beginners benefit most” is too broad: AI may lift output more when a task-specific gap leaves room for assistance. Whether they retain the advantage is separate, and depends on what the later unaided measure tests.

Sources: Does generative AI narrow education-based productivity gaps? Evidence from a randomized experiment; Generative AI at Work; Experimental Evidence on the Productivity Effects of Generative Artificial Intelligence

What does an assisted score actually tell you?

An assisted score measures the combined performance of a person, a tool, and the conditions of a task. It can show that a worker completed a memo faster or that a learner answered more questions correctly with help. By itself, it cannot show that the person can explain the reasoning, spot a faulty claim, reproduce the result, or handle a new case after access ends. Those are separate outcomes, and each needs its own measure. That distinction matters because “benefit” can refer to several things. Assisted task performance is the quality or amount of work produced while AI is available. Immediate comprehension asks what the person understands during or just after that work. Retention asks what remains after a delay. Near transfer tests whether someone can use learning on a similar task without the same assistance; far transfer would involve a meaningfully different problem. Persistence concerns whether people keep trying when work becomes difficult. Professional judgment concerns how they assess evidence, exceptions, and consequences in a real role. Workplace productivity concerns output over time in a workflow. These measures can move in different directions without contradicting one another. The randomized online study “Does generative AI narrow education-based productivity gaps? Evidence from a randomized experiment” illustrates the distinction. Its 1,174 adults, aged 25 to 45, completed a workplace-style problem-solving task with or without an AI assistant, then an unassisted module. The authors report that assistance improved the task for all education groups and narrowed the measured gap from 0.548 standard deviations without AI to 0.139 with it. In the later unassisted module, participants did not perform worse overall, and lower-education participants retained part of their improvement, though a sizable gap returned. The result supports partial carryover on this study’s task and follow-up. It does not establish broad workplace learning or how much skill would remain months later. A different outcome appears in the randomized experiment reported in “Borrowed or kept? How self-generation-first and direct-adoption AI workflows shape immediate unaided reasoning.” Participants were assigned to write with a self-generation-first workflow, adopt AI output directly, or work without AI, then complete a parallel essay without the tool 15 to 20 minutes later. In the primary analysis of 659 participants, the direct-adoption group scored lower than the no-AI group on reasoning quality: an adjusted difference of 0.38 points on a seven-point scale, described as a small effect. The self-generation-first group did not fall below the no-AI comparison. This is evidence about immediate near transfer in an essay task, not proof of lasting skill loss. The workflows also differed in effort, elaboration, and revision, so the study cannot attribute the pattern to one isolated ingredient. The paper “Generative AI without guardrails can harm learning: Evidence from high school mathematics” tests another boundary: students’ practice and exam performance in an educational setting. Its outcomes concern learning mathematics, not adult employees’ general capability. Read beside the workplace-style experiment, it shows why “the score went up” is incomplete reporting: the score may describe supported practice, while the later test asks what students can do independently. Imagine a client memo drafted with AI. A polished draft may improve assisted output, but it does not reveal whether its author can trace each claim to evidence, catch an exception, or explain why a recommendation follows. Those checks matter when the work is repeated or the tool is unavailable. When reading any result, ask what was measured, how soon after assistance, whether help remained available, and how closely the follow-up matched the first task. Do not turn a supported score into a claim of general expertise.

Sources: Does generative AI narrow education-based productivity gaps? Evidence from a randomized experiment; Borrowed or kept? How self-generation-first and direct-adoption AI workflows shape immediate unaided reasoning; Generative AI without guardrails can harm learning: Evidence from high school mathematics

What happens after access ends?

What happens when access ends depends on what the person was doing with the tool and what the later test asks them to do. A randomized online experiment with 1,174 adults aged 25–45 offers a positive result: participants completed a workplace-style problem-solving task with or without generative AI, then an unassisted module. In “Does generative AI narrow education-based productivity gaps? Evidence from a randomized experiment,” the researchers report that AI improved assisted performance for every education group, with larger gains among participants with less education. Once access was removed, treated participants did not perform worse overall than the no-AI group, and lower-education participants retained part of their improvement; a sizable education gap returned. This supports possible carryover, not proof of transferable workplace skill. One online task and follow-up cannot establish persistence over months or across job duties. Other controlled studies report weaker or unchanged immediate unaided performance. “AI Assistance Reduces Persistence and Hurts Independent Performance,” a randomized series involving 1,222 people across tasks including mathematical reasoning and reading comprehension, reports higher short-term assisted performance but worse unassisted performance and a greater likelihood of giving up after brief interactions of about ten minutes. This preprint does not establish workplace deskilling. In a high-school mathematics field experiment, “Generative AI without guardrails can harm learning: Evidence from high school mathematics” found practice scores rose with both a standard AI tutor and a tutor designed to give hints. On the later unaided exam, the standard-access group scored below the no-AI control, while the scaffolded-tutor group was not significantly different from control. This student learning study shows assisted and unaided scores can diverge, but does not establish a worker effect. Follow-up time also changes what “persists” can mean. “Experimental Evidence on the Learning Impact of Generative AI” reports a randomized, in-person study in which undergraduates studied an unfamiliar topic, wrote an essay with or without AI, and completed unaided assessments immediately and one week later. The abstract says test-score gains were smaller but still present after a week; they remained for students using AI as a tutor and faded for those having it write for them. This provides evidence of short-term learning carryover in a student assignment, but it is a preprint and does not measure recurring job performance. Its population, activity, assessment, and delay differ from the adult task, so the results are not competing estimates. A separate essay experiment narrows the role of interaction order. “Borrowed or kept? How self-generation-first and direct-adoption AI workflows shape immediate unaided reasoning” randomized 900 participants, with 659 in its primary analysis. After an AI-assisted argumentative essay, participants wrote a parallel essay unaided 15–20 minutes later. Those in the direct-adoption workflow scored lower than the no-AI group on reasoning quality, by 0.38 points on a seven-point scale (a small effect, d = −0.28); the self-generation-first group did not fall below that baseline. The workflows also differed in elaboration, revision, and effort, so no single cause is isolated. Its authors explicitly frame this as an immediate near-transfer result, not lasting de-skilling. Taken together, these findings allow both partial carryover and short-term losses: assistance is not automatically teaching or eroding skill. The outcome depends on the task, the user’s role in producing the answer, and the timing and content of the unaided check.

Sources: Does generative AI narrow education-based productivity gaps? Evidence from a randomized experiment; AI Assistance Reduces Persistence and Hurts Independent Performance; Generative AI without guardrails can harm learning: Evidence from high school mathematics; Borrowed or kept? How self-generation-first and direct-adoption AI workflows shape immediate unaided reasoning; Experimental Evidence on the Learning Impact of Generative AI

How can the same tool help in one design and hinder in another?

The same AI tool can raise immediate performance yet leave users less prepared to work unaided: access alone does not describe how people use it. Someone adopting a complete answer may finish with little practice diagnosing the problem. Someone who first forms a view, asks the tool to challenge or explain it, and checks the result still does some reasoning the task is meant to develop. This is a plausible practice explanation, not a settled mechanism; studies compare specific workflows and settings. In the randomized study “Borrowed or kept? How self-generation-first and direct-adoption AI workflows shape immediate unaided reasoning,” 900 participants were assigned to write an essay using a self-generation-first workflow, a direct-adoption workflow, or no AI; 659 were in the primary analysis. Fifteen to twenty minutes later, they wrote a parallel essay without the tool, scored blind for reasoning quality, idea diversity, and originality. The direct-adoption group scored lower than the no-AI group on reasoning quality, by 0.38 points on a seven-point scale, a small effect. The self-generation-first group did not fall below the no-AI comparison. This is immediate near transfer, not evidence of lasting skill loss. The workflows also differed in self-generation, revision, and effort, so the study cannot identify which ingredient explains the pattern. More time thinking, rather than workflow order alone, remains a credible alternative. A coding trial also separates task completion from learning. In “How AI assistance impacts the formation of coding skills,” 52 mostly junior software engineers learned an unfamiliar Python library in a randomized experiment. The AI group averaged 50% on a quiz about concepts, debugging, and code reading used moments earlier, compared with 67% for the hand-coding group. AI users finished about two minutes sooner on average, but this difference was not statistically significant. Within the AI group, stronger quiz scores appeared among people who asked for explanations or conceptual help while solving parts independently. Those patterns were not randomly assigned, so they are associations, not proven causes. The quiz followed one task by minutes; it cannot show whether a gap lasts or applies across software work. A classroom experiment shows why scaffolded assistance must be judged by its comparison. “Generative AI without guardrails can harm learning: Evidence from high school mathematics” followed nearly 1,000 students in about fifty ninth- through eleventh-grade classes at one Turkish high school over four 90-minute sessions. Against textbook-only practice, GPT Base and a structured GPT Tutor raised assisted practice scores by 48% and 127%. On a later unassisted exam, Base scored 17% below control, while Tutor was statistically indistinguishable from control, with no measured advantage. Tutor was instructed to give hints rather than direct answers and included teacher-provided solutions and common errors. This suggests design may affect the balance between assisted performance and unaided learning. It does not establish a worker effect: participants were school students doing mathematics with a purpose-built tutor. Together, these studies do not validate a prompt formula. They suggest a modest rule for recurring work: if independent judgment matters, preserve a human step that tests your understanding. Before requesting a finished client memo, for example, write down the issue and evidence that could change your view; then use assistance to test or organize that analysis, and verify the result. This is an illustration, not a tested intervention. A later comparable task without assistance can show whether you can explain the reasoning and handle an exception. Keep assisted quality and unaided capability separate: neither a polished deliverable nor one short quiz measures expertise. Structured conditions may also work through extra effort, task-specific guidance, or more time with material. Evidence from school lessons, essays, and a short coding exercise cannot establish that one workflow produces lasting gains in every job.

Sources: Borrowed or kept? How self-generation-first and direct-adoption AI workflows shape immediate unaided reasoning; How AI assistance impacts the formation of coding skills; Generative AI without guardrails can harm learning: Evidence from high school mathematics; Experimental Evidence on the Learning Impact of Generative AI

Does a longer workplace trial change the answer?

A three-month randomized trial provides a rare workplace test of both assisted output and later unaided judgment. “Does AI Assistance Enhance or Erode Expertise? Evidence from a Three-Month Field Experiment in Patent Drafting” studied 133 practicing patent lawyers at eleven U.S. intellectual-property firms using a custom drafting assistant. Draft quality on benchmark tasks improved at 10 days and 90 days, with reported effects of 0.34 and 0.38 standard deviations; junior lawyers had larger assisted gains. That establishes better performance with the tool in this setting, not that junior lawyers learned more. After three months, participants redlined an existing patent application without AI, a task the researchers describe as central to patent practice. Treated lawyers scored 0.32 standard deviations above controls, but this advantage was concentrated among senior lawyers. Junior lawyers had no average gain, although their scores shifted away from the middle toward both lower and higher outcomes. Thus the group with larger assisted drafting gains was not the group showing the average unaided judgment advantage. The researchers suggest foundational expertise may help people extract durable skill from assisted practice; this is an interpretation, not proof that seniority caused the difference. This working paper concerns a small, specialized group, one custom tool and a constructed judgment task scored by blinded patent attorneys. Three months of exposure is useful, but does not establish how every legal skill changes or predict an individual outcome. Its value is measuring production and later judgment separately. A different field experiment shows why those distinctions matter. “Task Expansion with Generative AI: Experimental Evidence on Middle-Skilled, Early-Career Workers” randomly assigned AI access for occupation-specific tasks to 673 final-year IT apprentices at twelve German vocational schools. Tasks included both curriculum work and problems deliberately beyond formal training. The Stanford Digital Economy Lab summary reports productivity gains across task types, with gains tending larger where unaided performance was lowest, and a 26–31 percentage-point increase on beyond-curriculum tasks. It also reports no reduction in immediate task comprehension and generally larger treatment effects among apprentices with above-median AI literacy. This is evidence that AI helped apprentices complete a wider range of tasks without lowering their immediate understanding. It is not evidence that they retained the capability after access ended: the accessible study summary does not report a later unaided performance test. Immediate comprehension and independent transfer are different outcomes. The apprentice experiment therefore complements, rather than settles, the patent-law result. One tests task expansion and contemporaneous understanding; the other includes unaided judgment after sustained workplace exposure. For a manager, expanded contribution can be useful while supervision remains necessary for exceptions and core reasoning. For a lawyer, a stronger assisted draft can coexist with the need to identify a weak claim without system support. These examples do not argue for withholding tools. They show why evaluation should begin by naming the desired result: task completion, quality, learning and independent judgment answer separate questions. “Generative AI at Work,” a study of 5,172 customer-support agents, supplies a production comparison. Access to a conversational assistant raised issues resolved per hour by 15% on average, with larger improvements for less-experienced and lower-skilled agents; the paper also reports evidence of learning in that deployed setting. Yet ongoing access at one firm is not a randomized withdrawal test of independent skill. Its learning evidence should not be equated with the patent-law trial’s later unaided judgment measure. Use assistance where it improves work, while checking capabilities needed when the system is unavailable, fails or meets an exception. Expanded output can be real even when long-term independent transfer remains unknown. If independent judgment matters, include a comparable unaided task in training or review and use the same quality criteria. It does not predict job security; it checks whether the workflow delivers both results and capability.

Sources: Does AI Assistance Enhance or Erode Expertise? Evidence from a Three-Month Field Experiment in Patent Drafting; Task Expansion with Generative AI: Experimental Evidence on Middle-Skilled, Early-Career Workers; Generative AI at Work

What should a worker measure on a recurring task?

To tell whether AI helps you become more capable, measure two outcomes separately: the work completed with the tool, and what you can explain or handle later without it. Compare one representative assisted attempt with a later, similar unaided attempt using the same quality criteria. Note what you diagnosed, changed and verified. This is a local check, not a validated test of durable mastery or job security. Choose a recurring, low-consequence task, such as drafting an internal summary from approved source material. Before starting, define what a satisfactory result requires: accurate coverage, clear reasoning and any relevant time limit. Follow workplace rules about tools and data. During the assisted attempt, record where you accepted, revised or rejected suggestions and how much checking remained. Later, try a comparable task without the tool. Can you explain key choices, reproduce the main steps and notice an important exception? Keep task difficulty similar; one unusually easy or hard assignment can mislead. The measures should stay distinct. Record whether assisted work met your standard, how long it took and what corrections remained. For unaided work, note whether you can explain the reasoning and handle an exception. A faster assisted result does not establish independent competence; a slower unaided result does not prove that no learning occurred. In “Does generative AI narrow education-based productivity gaps? Evidence from a randomized experiment,” 1,174 adults aged 25–45 completed a workplace-style problem-solving task followed by an unassisted module. The abstract reports that AI improved task performance, participants with less education retained part of their improvement, and a sizable gap re-emerged. Better follow-up performance with intensive AI use appeared only alongside sustained effort. This single online task does not establish general skill acquisition or months-long retention. The interval changes what a check can establish. “Borrowed or kept? How self-generation-first and direct-adoption AI workflows shape immediate unaided reasoning” tested a parallel essay 15–20 minutes after different AI workflows. Direct adopters scored lower on unaided reasoning than the no-AI comparison; the self-generation-first group did not fall below it. The authors call this immediate near transfer, not lasting de-skilling, and the workflows differed in effort, revision and elaboration. A next-day task asks something different. If you find a gap, practice the specific step you could not explain, or ask a knowledgeable colleague for feedback. If results are mixed, repeat the check before a larger decision; fatigue and task variation can affect one attempt. For high-consequence work, follow established review procedures rather than withholding required tools or oversight. The three-month patent-law field experiment, “Does AI Assistance Enhance or Erode Expertise? Evidence from a Three-Month Field Experiment in Patent Drafting,” reports assisted drafting gains and a later unaided judgment advantage concentrated among senior lawyers. It supports checking independent judgment where it matters, not inferring career prospects from one task.

Sources: Does generative AI narrow education-based productivity gaps? Evidence from a randomized experiment; Borrowed or kept? How self-generation-first and direct-adoption AI workflows shape immediate unaided reasoning; Does AI Assistance Enhance or Erode Expertise? Evidence from a Three-Month Field Experiment in Patent Drafting

A bounded next step

For your next recurring task, decide what you need from AI: faster assisted output, stronger independent capability, or both. If output alone is the goal, judge the assisted result against the task’s quality and review requirements. If independent capability matters too, keep meaningful reasoning in your hands and check whether it carries over later without the tool. The evidence does not support a rule that the largest assisted gain predicts the largest lasting skill gain. “Does generative AI narrow education-based productivity gaps? Evidence from a randomized experiment” found larger assisted gains among less-educated participants on a workplace-style problem-solving task, with partial carryover for that group. “Borrowed or kept? How self-generation-first and direct-adoption AI workflows shape immediate unaided reasoning” reports a different result: after brief interaction, direct adoption was followed by lower performance on a parallel unaided writing task, while the self-generation-first group did not score below the no-AI group. That comparison came 15–20 minutes later and cannot establish durable learning. For one low-consequence task, set quality checks before you begin, note what the tool contributed and what you corrected, then try a similar task later without assistance. Compare your explanation and handling of exceptions as well as the finished product. One attempt can guide practice or feedback; it does not prove mastery or justify a credential or career change. Task design and the follow-up measure could change your conclusion. The free task checker can help you inspect occupational change pressure at task level; its signals are not a validated probability of job loss.

Sources: Does generative AI narrow education-based productivity gaps? Evidence from a randomized experiment; Borrowed or kept? How self-generation-first and direct-adoption AI workflows shape immediate unaided reasoning

Questions readers ask