What Does Anthropic’s Index Show About AI Learning and Task Success?

What does the March report establish—and what is its answer?

Anthropic’s *Anthropic Economic Index report: Learning curves* finds that Claude accounts opened at least six months earlier had a four-percentage-point higher conversation-success estimate after measured controls. The adjusted association is consistent with users learning to work with AI, but does not show that the same people improved through practice. This February 2026 snapshot of Claude use is not a forecast of any knowledge worker’s job prospects. It is a reason to inspect one recurring workflow: where assistance helps, what review remains necessary, and whether the finished work meets its quality bar. Adjustment makes the result more informative than a raw contrast, since the association remains after measured task, request, and context differences are considered. But it cannot show that account age caused better performance or distinguish practice from early adoption, continued use, or unmeasured factors. “Learning curve” is a plausible interpretation of a group pattern, not an individual trajectory. A worker can test the practical question on a real task by including checking, correction, and sign-off in the assessment. The report does not answer that question for a particular role or show whether employers will adopt tools, redesign roles, or reduce staffing.

Sources: Anthropic Economic Index report: Learning curves

What exactly did Anthropic compare?

Anthropic’s *Anthropic Economic Index report: Learning curves* compares conversations drawn from its own Claude services during a defined week, then groups those records by the age of the account associated with them. The report says it analyzed a one-million-conversation sample from Claude.ai and first-party API use collected from February 5 through 12, 2026. This is a large set of service interactions, but its basic observation is a conversation in Anthropic’s products. It is not a direct observation of a worker completing a job task, and the number of conversations should not be read as the number of distinct workers or users.

The distinction matters because a single account can generate multiple conversations in the sampling window. Those records can describe a broad range of requests and make task-level comparisons possible, while giving frequent users more than one appearance in the data. The report’s comparison therefore concerns conversations associated with accounts of different ages; it does not follow one million people from first use through later use. Nor does the record, as described here, establish how much each account holder practiced, whether their work skills changed, or whether any change carried over to tasks outside Claude.

Anthropic defines the higher-tenure group by signup date: an account counts as high-tenure if it was created at least six months before the February data pull. The cutoff is an operational grouping rule, not a measure of six months of continuous use. Someone who opened an account earlier may have used it heavily, intermittently, or very little; signup age alone does not tell the analysis which. Conversely, a newer account could belong to a person with extensive experience using other AI tools. Account tenure is thus a property of the service relationship, not a verified record of a person’s AI-learning history.

To organize the requests, Anthropic maps conversation content to occupational task categories using O*NET, the U.S. occupational information framework, and groups related requests into task clusters. These labels let the report compare similar classified work across account-age groups rather than treating every prompt as interchangeable. But a classified conversation remains a representation of a request: it does not establish that the user holds the occupation associated with the mapped task, performs that task as paid work, or submitted the resulting output to an employer. A request about a task and a worker’s complete role are different units of analysis.

The source describes the analysis as privacy-preserving aggregation. That is relevant to how Anthropic reports patterns in product use, but aggregation does not convert the records into a representative sample of the workforce. The conversations come from people who chose to use Claude.ai or an API built on Claude, under the access and product conditions of those services. The evidence can speak to patterns in sampled Claude interactions and how those patterns differ by account tenure; it cannot by itself establish how common the same behaviors are among workers who do not use these products or use other systems.

The comparison’s unit is sampled Claude conversations associated with newer and older accounts during one week. It is not a worker panel, representative employee survey, or employer study of staffing and job design. The classifications describe use in Anthropic’s product ecosystem at a particular moment. Claims about typical employees would need a worker-based sample; claims about improvement would need repeated observations of people doing comparable work.

Sources: Anthropic Economic Index report: Learning curves

Why is account tenure not the same as a learning curve?

Anthropic’s *Anthropic Economic Index report: Learning curves* compares different account groups at one point in time; a learning curve, in the stronger sense, describes change within the same people as they gain experience. The report’s tenure measure is account age: whether an account was opened at least six months before the sampled period. That makes signup date a useful way to sort conversations, but not a record of what any user did between signup and observation. An account may be old while its owner used Claude rarely, or newer while its owner brings substantial practice with other AI systems. The observed gradient can therefore be consistent with learning without directly measuring learning. To establish within-person improvement, the analysis would need linked observations of the same people’s performance across time, with meaningful information about exposure and comparable work. The current cohort comparison answers a different question: how do sampled conversations associated with older and newer accounts differ?

Several processes could generate a difference between those cohorts. People who adopted Claude earlier may have been more technically curious or comfortable experimenting at the outset. If so, an apparent advantage at longer tenure could partly reflect baseline differences that predate account creation, rather than a gain produced by use. The report’s tenure clock starts with opening a Claude account; it does not establish when someone first encountered generative AI, what prior tools they used, or what their work required them to learn. Those distinctions matter because prior familiarity could affect both adoption timing and subsequent interactions. This is a plausible cohort explanation, not a finding that early users were more skilled. The evidence shows the grouping and the outcome pattern; it does not independently measure every user’s starting ability. Treating the account date as a personal starting line would quietly assume the very history needed to demonstrate development.

Continued use also changes who appears in an older-account group. Anthropic’s report identifies survivorship as one possible composition effect: among people who opened accounts earlier, those who still use the service may be disproportionately people who find it valuable or have work suited to it. Users who stopped may be less visible in the later sample, while persistent users contribute conversations. This possibility does not invalidate the remaining users’ records or justify dismissing the association. It changes the interpretation of the group: observed older accounts can represent a selected set of continuing users, not everyone who began at that time. A favorable pattern among those still present may combine individual development with differences in who persists. Without information on nonusers and departures, the report cannot assign how much each process contributes. Survivorship is thus a reason to describe the comparison carefully and to track who leaves in future evidence, not a reason to discard a result that still describes sampled conversations.

Anthropic also reports gradients in characteristics inferred from the prompts. In its binned comparison, the education level associated with prompts rises by almost one year for each additional year of account tenure; the share of conversations classified as personal use is lower among longer-tenure cohorts than among the newest accounts. These patterns add texture to the cohort contrast, but they are not a transcript of a user’s education or a measure of acquired competence. The education figure is associated with the kinds of tasks represented in prompts, so a shift could arise because users bring different work and requests, because task mix changes over tenure, or because the same people develop. Likewise, a changing share of personal-use conversations describes the sampled request mix, not a direct measure of each person’s goals. These are signals that account groups differ in what appears in the service. They do not identify which users changed or why. The report’s own interpretation leaves both learning and selection in view.

Access and product conditions provide another route to group differences. Account age alone does not encode which Claude interface a conversation used, which model or features were available, or whether a user had access to a paid plan or workplace-supported workflow at the relevant time. Product availability can change after signup, and people can move between ways of using a service. If older and newer account conversations differ in those conditions, then exposure to capabilities or interaction opportunities may contribute to their observed outcomes. That would be a difference in use environment, not automatically a gain in transferable skill. The report can adjust for some recorded context in its regressions, but this section’s central point is about what the tenure variable itself means: elapsed time since account creation bundles together many possible histories rather than isolating practice. A user who learned a specific interface may also not have learned a general method that transfers to unfamiliar models, tasks, or workplace standards.

Learning-by-doing remains a plausible interpretation: practice could help users frame requests, recognize weak answers, supply context, and decide when to verify. But initial differences, changing request mix, or continued-use selection could produce the same pattern. Tenure does not measure practice, learning rate, or transfer. A convincing learning curve would link a person’s practice to improvement on comparable work and test transfer to other tasks. Until then, the report offers an association to investigate, not a measured personal trajectory.

Sources: Anthropic Economic Index report: Learning curves

How much does adjustment change the tenure association?

Anthropic’s *Anthropic Economic Index report: Learning curves* says the raw comparison showed high-tenure accounts with a 10% higher conversation-success rate. That is a relative difference as the report describes it: the estimate is 10% higher than the comparison group’s rate, not an increase of ten percentage points. In its fully controlled model, Anthropic reports a four-percentage-point difference. Percentage points describe the arithmetic gap between rates; for example, a hypothetical move from 60% to 64% is four percentage points, though it is about 6.7% higher relative to 60%. The report’s four-point figure should therefore not be restated as a four-percent increase. The two estimates answer related but distinct descriptive questions: how far apart are the groups before adjustment, and how far apart are they after accounting for the listed measured differences? The latter remains positive, but its interpretation is still an adjusted association between account cohorts.

The controls are added in stages to make the comparison less sensitive to differences in the conversations being grouped. The report first accounts for task and request-cluster differences, using fixed effects for the O*NET task categories and request clusters represented in the data. In plain language, this asks whether the tenure association persists when comparisons give weight to differences between types of work requests and clusters, rather than allowing a group’s task mix alone to explain the result. A fixed effect is a statistical adjustment that absorbs average differences associated with a category. It does not mean every task is identical or that each conversation has been experimentally matched. The model then adds recorded context for the Claude model, use case, and country. These steps narrow the set of straightforward measured explanations, while leaving the outcome and tenure measure as observational quantities from the report’s sample.

The sequence matters because an unadjusted group gap can reflect what people ask the system to do as well as how well it handles those requests. If older accounts are associated with a different distribution of task categories or contexts, a raw rate could partly mirror those differences. Adding task and request-cluster effects tests whether the association disappears when that measured composition is absorbed. Adding model, use-case, and country effects then asks whether the result can be accounted for by those recorded categories as well. Anthropic reports that the association remains and is four percentage points in the fully controlled specification. This is evidence against a simple account in which the observed difference comes solely from the measured task/request mix and listed context variables. It is a more disciplined descriptive comparison than the raw figure, while still depending on the variables and model choices that the report specifies.

A four-percentage-point adjusted association is meaningful in that limited sense: the measured differences do not erase the pattern. Adjustment does not mean the two cohorts became equivalent in every relevant way. Fixed effects control average differences across recorded categories; they do not create random assignment, verify that all relevant features were captured, or turn account age into an amount of practice. Nor does the estimate say that every older-account user is four points more likely to succeed, or that one person’s success rate would rise by four points after six months. It is a model-based group contrast conditional on included variables, not an individual treatment effect. The distinction is useful for reading the report fairly: the result is not merely a raw task-mix artifact on the measured dimensions, yet it cannot be read as a direct estimate of what a worker would gain by practicing with Claude.

Statistical significance, where reported, addresses compatibility with sampling variation under the model; it does not establish why the association exists. Adjustment improves specificity but does not identify changes caused by learning. The report frames the evidence as consistent with learning-by-doing while leaving alternatives unresolved. The adjusted result survives specified comparisons and supports a pattern in this dataset; its source remains uncertain. That is the boundary between saying the tenure association persists and saying practice caused skill growth.

For a reader, the four-point estimate can sharpen the question without supplying a personal forecast. It suggests that in Anthropic’s sampled conversations, measured task and context differences do not account for the entire success gap between tenure groups. It does not tell a worker whether their own tasks, tools, experience, or review criteria resemble those conversations. The report’s success measure is also the one defined by its analysis, so this coefficient should not be translated into a four-point improvement in workplace quality, productivity, or job performance. Those outcomes require their own observations. The practical value of the adjusted estimate is that it keeps the learning hypothesis plausible after specified controls while leaving the mechanism open. A worker evaluating a workflow still needs to judge the actual deliverable against its acceptance standard and include checking effort; this regression alone cannot decide whether a particular use is worthwhile.

Sources: Anthropic Economic Index report: Learning curves

What changes in the conversations of longer-tenure users?

Anthropic’s *Anthropic Economic Index report: Learning curves* describes longer-tenure conversations as more collaborative in form: users were more likely to iterate with Claude and less likely to use directive exchanges that specify a task and expect a bounded response. The report also found that the high-tenure group was seven percentage points more likely to use Claude for work. These are differences in the conversations Anthropic classified, not a record of a person changing their habits after six months. They show that the tenure contrast has a behavioral texture beyond the success estimate: the groups appeared to bring different purposes and interaction styles to the service.

Iteration means that an exchange contains follow-up turns through which a user can refine the request, supply missing context, question a response, or ask for a revision. That pattern can fit work where the desired result is not fully specified at the start: drafting, analysis, or a problem whose relevant constraints emerge during exploration. A directive exchange can fit a more bounded subtask, such as transforming clearly specified material into a predictable format, provided the output can be checked. These are interpretations of why interaction forms may differ, not task-level causal effects estimated by the report. The same number of turns can also contain substantive evaluation or unproductive repetition; turn count alone cannot reveal which.

This makes the interaction finding relevant to how experience might be expressed. A practiced user may know when to let a system attempt a bounded step and when to intervene, clarify, or inspect an answer. More turns could therefore reflect active steering rather than simply delegating a larger share of work. But the report does not identify a sequence in which one person first delegated, then learned to iterate, and consequently succeeded more often. The pattern is equally compatible with users who already prefer collaborative exchanges, or with the high-tenure group encountering tasks that invite back-and-forth. Iteration is a plausible marker of how users manage a task, not a demonstrated mechanism for the tenure association.

Anthropic’s report adds two indicators of task mix. The education level associated with prompts rose by almost one year for each additional year of account tenure, and the ten most common tasks accounted for 20.7% of high-tenure conversations compared with 22.2% of low-tenure conversations. The education figure refers to an estimate associated with prompt content, not a survey of users’ qualifications. The concentration figures indicate a modestly broader distribution of classified task types among longer-tenure conversations. Together with the higher work-use share, these results suggest that the older-account sample included a somewhat different and less concentrated set of requests. They do not establish whether people broadened their work, whether a more varied set of users remained, or whether task selection changed as they became familiar with Claude.

Iteration may help with ambiguous tasks or examining alternatives; concise delegation may suit stable requests with straightforward verification. Neither style is inherently more skilled. Extra turns take time and add material to review, while short exchanges can leave assumptions unchecked. The useful question is whether clarification improves the deliverable enough to justify interaction and checking. Anthropic’s findings make this a work-design question, but do not show that more prompting is always better or that copying longer-tenure users’ style improves results.

Sources: Anthropic Economic Index report: Learning curves

What does Anthropic mean by conversation-level task success?

Anthropic’s *Economic Index report: Economic primitives* uses “task success” for an outcome its classifier assigns to a conversation: whether Claude appears to have successfully completed the task represented by that exchange. A classifier is a model used to label conversation properties from a specified prompt. In this case, the labeled outcome gives Anthropic a consistent way to summarize many sampled interactions. It is a conversation-level estimate of task completion under the report’s operational definition, not an independent inspection of work products against an employer’s standards.

In its January 2026 analysis, Anthropic estimated task success at 67% for sampled Claude.ai conversations and 49% for sampled first-party API conversations. Those values describe separate product samples and the report’s classification method. They are not pass rates from a shared exam or a randomized comparison of interfaces. The API records were treated as isolated input-output pairs, whereas Claude.ai records could preserve multi-turn exchanges. Users, task composition, and the available interaction record therefore differ alongside the platform label. The figures provide an aggregate description of the sampled traffic, but the gap cannot by itself show that one interface causes more successful work.

The word “success” can sound broader than the measured label. A conversation may appear to reach its requested endpoint while still containing a factual error, omitting a required component, violating a policy or format, or requiring substantial human repair. Conversely, an exchange classified as incomplete might still provide a useful partial result. The reported estimate is not a separate audit of factual accuracy, completeness, compliance, user acceptance, or downstream consequences. Those properties can matter differently by task: a draft may be useful after editing, while a calculation, client communication, or regulated decision may require stricter checks before anyone can rely on it.

A workplace measure would need to make its acceptance criteria explicit and assess the deliverable at the relevant point in the workflow. For factual work, that might include checking claims against approved records; for a structured output, it might mean confirming every required field and rule; for a recommendation, it could include review by the person accountable for the decision. These examples describe possible evaluation standards, not measures Anthropic applied to the January sample. A fuller assessment could also record revision burden, time spent checking, whether the recipient accepted the result, and what happened after it was used. Such outcomes answer different questions from whether a classifier judges that the conversation’s task was completed.

The 67% and 49% estimates are aggregate signals within Anthropic’s analysis, not shares of a worker’s tasks that will meet a quality bar or evidence that an organization will save time. Local evaluation must compare the deliverable with its standard, including review and correction. The classifier outcome remains informative at its stated level: a model-labeled conversation result, not verified workplace performance. Each platform rate reflects its own sampled tasks and users, so the percentages are not a head-to-head product score or a basis for redesigning a job.

Sources: Economic Index report: Economic primitives

Why do interface and task-duration measures change the comparison?

Anthropic’s *Economic Index report: Economic primitives* presents the Claude.ai and API figures as descriptions of different sampled records, not as a test that holds interaction format constant. The January 2026 analysis estimated average human-alone task duration at about 3.1 hours for Claude.ai conversations and 1.7 hours for first-party API traffic. It separately estimated human-AI interaction time at about 15 minutes and 5 minutes, respectively. The first pair describes estimated time a person would take to do the task alone; the second describes estimated time spent interacting with AI. They are distinct primitives, with different meanings and denominators. Neither pair is a direct stopwatch comparison of matched workers completing identical assignments.

The record structure helps explain why the samples differ. Anthropic’s Claude.ai data include full multi-turn transcripts, so an exchange can preserve a user’s initial request, the model’s reply, follow-up clarification, correction, and revision. In the API sample, the report treats request-response records as isolated pairs because prior exchanges are not linked in the data. A pair can show a request and answer, but it cannot necessarily reveal whether the request began a longer exchange, followed omitted context, or was one step in a larger workflow. This is a difference in what the dataset makes observable as well as in the product channel being measured.

That distinction matters when interpreting human-AI time. A multi-turn record may contain several visible rounds of work and therefore more recorded interaction than an isolated pair. But the five-minute API estimate should not be read as evidence that API users needed less total assistance: linked earlier or later work may not appear in the same record. Likewise, the 15-minute Claude.ai estimate does not show that added turns improved the result. The report’s interaction measure characterizes estimated time with AI in the sampled task exchange; it does not independently establish the value of each turn, the amount of unrecorded preparation, or the time spent checking the final output.

Human-alone duration and interaction time should also not be subtracted mechanically to claim a net saving. One is an estimate of the time needed to perform a task without AI; the other is an estimate of time spent interacting with AI in the sampled exchange. A valid workflow comparison would need a common task boundary and the time spent preparing inputs, evaluating the response, making corrections, and finishing the deliverable. If those activities occur outside the transcript, a transcript-based interaction estimate does not by itself include them. These figures are useful for describing what the report estimates about each sample, but the arithmetic alone cannot establish how much a worker or organization saves.

The longer estimated human-alone duration for Claude.ai tasks also changes the comparison’s composition. If sampled Claude.ai conversations concern work that people would otherwise spend more time doing, their time estimates can exceed the API average even if the interface itself has no effect on task duration. Differences in request mix, user populations, task boundaries, and available context travel alongside the channel label. The two platform averages therefore combine characteristics of the sampled work with the way each product’s records capture interaction. Anthropic’s estimates do not isolate a platform effect by matching tasks or randomly assigning users to interface types.

A longer exchange could let a user clarify requirements or correct an assumption, but it could also reflect task complexity, user preference, or differences in sampled work. The averages do not establish whether added interaction raised success. The comparison suggests that interaction format and context may matter; testing that requires comparable tasks and consistent definitions of human-alone and human-AI time.

Sources: Economic Index report: Economic primitives

How do task complexity and success pull in different directions?

Anthropic’s *The Anthropic Economic Index report: New building blocks for understanding AI use* describes a pattern in its November 2025 Claude sample: estimated speedups were larger for tasks associated with higher schooling requirements, while estimated task success was somewhat lower on those harder tasks. In the report’s Claude.ai comparison, estimated speedup was about ninefold for tasks associated with high-school-level input education and twelvefold for tasks associated with college-level input education. Estimated success was 70% for tasks associated with less than high-school-level understanding and 66% for those associated with college-level understanding. The estimates describe different dimensions of sampled work, so the larger speedup does not imply that harder tasks were more likely to be completed successfully.

The education labels are proxies inferred from task descriptions, not records of the users’ credentials or measures of the knowledge a particular worker needs. Anthropic uses them to characterize the apparent level of input education required by tasks in the sample. The categories therefore summarize task mix at an aggregate level. A higher associated education level can correspond to work that takes longer when done by a person alone, leaving more estimated time available to save when AI assists. That does not mean the system independently handles every difficult component, nor does it establish that a credential causes greater benefit. The comparison is about estimated primitives grouped by inferred task characteristics.

The two patterns can coexist because speedup and success answer different questions. A task may take a long time for a person to complete alone, and an AI-assisted exchange may still reduce some portion of that time, producing a large estimated ratio. Yet the system may fail to finish the task to the report’s success criterion more often than it does on simpler work. Partial progress can be fast while leaving unresolved reasoning, checking, or completion. A high speedup estimate therefore describes a time relationship under the report’s approach; it is not a measure of the share of work delivered at acceptable quality or of the value of the result.

This also explains why average time saved before quality adjustment can overstate useful productivity. If the estimate counts assistance on attempts that do not reach the task endpoint, time associated with a fast but incomplete result cannot automatically be counted as time saved on a successfully completed deliverable. In a real workflow, someone may need to identify omissions, repair reasoning, rerun the task, or complete the work manually. Those costs can narrow or eliminate the apparent advantage. Conversely, a partial output can still help if it reliably completes a useful subtask and review is manageable. The task’s acceptance standard and the point at which time is counted determine whether assistance translates into a practical gain.

The report’s estimates are model-based summaries of conversations and task characteristics, not observations of organizational throughput. Throughput would require knowing how many acceptable deliverables a team produces over a period, with rework, review, coordination, workflow delays, and any changes in demand included. A task-level speedup ratio does not capture those system effects. Nor can the four success percentages be translated into a forecast for a profession or worker. They show that estimated time and estimated completion do not move in lockstep across these task groups, which is precisely why one should not judge assistance by speed alone.

Lower estimated success on the more demanding group does not mean AI has no useful role there. If assistance produces a verifiable first draft, a bounded analysis step, or another partial contribution that reduces effort without lowering the final standard, it may be worthwhile even when full task completion is less frequent. That judgment depends on the particular work product, the cost of review, the consequences of an error, and whether the remaining steps fit the workflow. These are conditions for evaluating a use, not outcomes measured by Anthropic’s aggregate estimates. The careful conclusion is narrower: apparent speedups can be larger on complex tasks while success is lower, so a worker or organization needs both quality and total-workflow time to decide whether the assistance is productive.

Sources: The Anthropic Economic Index report: New building blocks for understanding AI use

What can METR’s task horizon add—and what does it measure?

METR’s *Task-Completion Time Horizons of Frontier AI Models* gives a capability benchmark a human-time unit: the estimated duration that an expert would need to complete a task. Its “time horizon” is not how long an AI system runs by itself, nor a forecast of how much of a job it can take over. METR estimates each task’s human duration, tests model agents on tasks with explicit completion criteria, and fits a relationship between task duration and the probability of success. The horizon is the duration at which that fitted probability reaches 50%. It is therefore a threshold on a benchmark curve, not a typical completion time, a promise that a model will finish half of everyone’s work, or a measure of the fraction of a workday automated.

The distinction between task duration and autonomous elapsed time matters. Human duration estimates how long an expert would need to do the task; the agent may take a different amount of clock time, use tools, or fail before finishing. A task estimated to require several hours of expert work does not imply the model can sustain several hours of reliable operation. METR’s threshold asks how task duration relates to completion probability across the tested suite. Tasks just below the horizon can still fail, and some longer tasks can succeed. The single threshold compresses a distribution into a comparable summary; it does not describe every model attempt or the quality of every output.

The suite makes measurement possible by concentrating on work with clear boundaries and checkable endpoints. METR’s methodology page describes tasks selected largely from software development, machine learning, and cybersecurity, with substantial representation of software work. The task authors specify success criteria so evaluators can determine whether an agent completed the requested work. Human expert duration is established through task-specific estimates and calibration, rather than by assuming that a task’s apparent complexity corresponds to a fixed number of work hours. These design choices help compare model performance across tasks whose endpoints can be assessed. They also define the benchmark’s population: evaluated agents on a curated set of tasks, not a representative sample of occupations or workers.

That makes the horizon informative for a particular question: how does an agent’s chance of completing well-specified, low-context tasks vary as the estimated expert effort rises? Low-context tasks limit dependence on a long organizational history or information available only through colleagues, internal systems, tacit norms, and changing priorities. Many jobs contain precisely those dependencies. Even a bounded software task can differ from production work when a real codebase has hidden constraints, a reviewer must judge maintainability, or a change has consequences beyond passing a test. A benchmark completion criterion can establish that the specified endpoint was reached; it does not automatically establish that the result met an organization’s broader quality, security, coordination, or accountability requirements.

The fitted 50% point should also be read as a property of the evaluation design. It depends on which tasks enter the suite, how expert duration is estimated, what counts as completion, and how the probability curve is fit. A shift in any of these can move the reported horizon even if the headline remains a single time value. This does not make the benchmark arbitrary: consistent definitions allow meaningful comparisons within the method. It means that the number travels with its task set and protocol, rather than serving as a context-free unit of general intelligence or work capacity.

Compared with Anthropic’s conversation-level analysis, METR is measuring a different object. Anthropic’s tenure comparison concerns observed user accounts and classified conversation outcomes; METR’s horizon concerns agent performance on a controlled task suite indexed by expert duration. One can inform questions about how AI users’ observed task success varies with account tenure; the other describes capability on tasks whose duration and completion can be measured. Neither instrument substitutes for the other. A controlled benchmark can reveal capability changes where measurement is tractable, but a task that resembles a workday in estimated expert duration does not make the result generalizable to all work. For a worker, the useful translation is to identify whether a local task shares the benchmark’s clear endpoint and limited context, then evaluate the real deliverable, review burden, and surrounding dependencies directly.

Sources: Task-Completion Time Horizons of Frontier AI Models

What does the historical capability trend add to the learning question?

METR’s *Measuring AI Ability to Complete Long Tasks* reports that the 50%-success time horizon on its evaluated tasks doubled roughly every seven months between 2019 and early 2025. The result summarizes a sequence of model evaluations on the paper’s benchmark suite: as the tested models improved over that period, the estimated duration of tasks they could complete at the 50% threshold rose at that approximate rate. This is evidence of rapid capability progress on those measured tasks during that dated interval. It does not mean every seven months a model becomes twice as useful in a workplace, or that the amount of work automated doubles on a fixed schedule.

The seven-month figure comes from fitting a trend to benchmark horizon estimates over time, not following a group of people as they learn to use AI. The observation unit is model performance on evaluated tasks, with task duration and completion criteria supplied by the benchmark method. The trend therefore concerns changes across successive systems. Anthropic’s account-tenure finding, by contrast, compares conversations from newer and longer-tenure accounts within its usage sample. Account age is not a controlled practice period, and a difference between account groups does not itself trace a person’s learning curve. Putting the results side by side clarifies the levels: one is about model capability over calendar time; the other is about observed user-group differences at a point in time.

These processes could interact in actual work without being the same process. A more capable model may alter which tasks a user attempts, how much supervision a workflow needs, or what practice is available to the user. A user may also become more effective through familiarity, training, or better task selection while the underlying model changes. But METR’s trend does not measure user learning, and Anthropic’s tenure association does not measure successive-model capability. Neither study’s result establishes how much of any workplace change comes from the other process. Separating them would require evidence that observes model versions, user experience, task conditions, and independently assessed outcomes together, rather than treating benchmark progress as an explanation for an account-tenure difference.

A trend rate can also obscure variation around the fitted line. Individual models do not necessarily advance by the same amount, and a single benchmark threshold does not tell a reader whether gains came evenly across task types or concentrated in a subset. The reported doubling is a useful summary of the paper’s measured trajectory, not a claim that every point on the suite improved at a uniform pace. This is another reason to preserve the date range and evaluation scope when using the result: a trend summary supports comparison over that study’s observations, while a current task decision needs current evidence on the specific work and system.

The historical estimate covers the tasks, models, duration estimates, and methods evaluated through early 2025. Any extension depends on whether progress on METR’s suite generalizes; it is not a forecast of workplace capability. The benchmark is strongest for self-contained tasks with consistent evaluation. Applying its pace to work with changing context, team dependencies, ambiguous criteria, or costly errors adds assumptions the trend does not test. METR’s current time-horizon page uses an updated suite and reports changing values, illustrating that horizons depend on suite and date. Those values should not be merged with the paper’s 2019–early-2025 estimate as one unchanged series.

For someone deciding whether to learn a tool or redesign a task, the trend supports urgency about checking local workflows, not a prediction about personal job security. Fast progress on a benchmark can make an old evaluation stale: a workflow that failed with one model may warrant a new test with a later system, and a workflow that passed should still be checked against current quality and review requirements. That is a practical inference from a changing capability measure, not a METR labor-market finding. The seven-month doubling does not indicate hiring demand, employer adoption, task substitution, or displacement. It cannot confirm or contradict whether longer-tenure users learn to use Claude more successfully; it instead shows why model capability and user learning need separate evidence and separate dates.

Sources: Measuring AI Ability to Complete Long Tasks; Task-Completion Time Horizons of Frontier AI Models

What future evidence would make the learning explanation stronger?

The clearest test would follow the same people over time and measure what they actually do with AI, rather than sorting accounts only by signup date. Anthropic’s *Economic Index report: Learning curves* identifies selection, early-adopter sophistication, task choice and survivorship as possible explanations for its tenure pattern. A follow-up should therefore record each participant’s active use, practice frequency, training received and workflow support, alongside periods when they do not use the tool. Signup age is a poor substitute for those exposures: two people with equally old accounts may have very different amounts of practice. This is a proposed evidence standard, not a description of Anthropic’s current method.

Repeated observation would help distinguish improvement within a person from differences between people, but it needs comparable work to make the change interpretable. Researchers could ask participants to complete recurring tasks, or matched tasks with similar input, difficulty and stakes, at several points. The task should be described well enough to see whether its requirements changed; where work itself evolves, the changes should be logged rather than treated as noise. Researchers should also record the model and interface version, dates, and any major workflow change. Otherwise, a better result could reflect a model update, an easier task, or new assistance as readily as accumulated user skill. Comparing a person’s own trajectory with matched tasks would narrow these explanations without pretending that ordinary work stays fixed.

The outcome measure should be independent of the tool provider and tied to the work’s real acceptance criteria. For each output, trained reviewers who do not know the participant’s tenure or study condition could score accuracy, completeness, policy compliance and usefulness against criteria set before review. Where possible, multiple reviewers and a record of disagreements would show how stable the judgment is. The study should separately capture time spent prompting, checking, correcting and completing downstream steps; an acceptable draft is not necessarily a useful workflow if verification absorbs the apparent saving. These measures would let researchers ask whether people improve at producing accepted work, whether they become faster without losing quality, and whether any gain survives the review burden. A conversation classified as successful cannot answer all three questions by itself. Acceptance thresholds should be explicit before the task is run: for a document, for instance, they might specify required fields and permitted error types, while the responsible workplace owner defines which errors are disqualifying. That prevents reviewers from quietly changing the standard after seeing a fluent answer.

A randomized comparison could strengthen the causal claim for one part of the process. For example, eligible participants might be assigned to structured practice, a defined training sequence, or workflow support, while a comparison group receives ordinary access; both groups could then perform the same repeated tasks under the same evaluation rules. Random assignment would help separate the effect of that specific intervention from prior differences, although it would not isolate every kind of informal learning. Researchers should report who leaves the study, which observations are missing and whether attrition differs by group or early performance. Finally, a transfer test should use an unfamiliar task, not just a rehearsed one. If improvement appears on trained tasks but not a new task, evidence would favor task familiarity over a broader skill gain. A study could also compare outcomes with and without AI on a subset of the same task types, so any measured benefit is set against unaided performance rather than a before-and-after impression alone.

Confidence in learning-by-doing would rise if practice dose predicted within-person improvement on independently scored, repeated work; if quality held while total completion and review costs changed favorably; and if some benefit transferred to unfamiliar tasks. The case would be stronger if a randomized practice or support condition produced a similar advantage and the pattern remained after accounting for model versions, task changes and missing participants. A null result, gains limited to one narrow task, or improvements explained by faster models would weaken that interpretation. Even a well-run study would establish a bounded effect for its people, tools and work conditions, not a universal learning curve across occupations or workplaces.

Sources: Anthropic Economic Index report: Learning curves

What is a proportionate next step at work?

Choose one recurring task that workplace policy permits and whose consequences are low enough for a careful trial. Before using assistance, write down the task’s existing quality bar and how long the full process normally takes. On a bounded instance, use an approved tool, then record the time spent checking and correcting as well as the final completion time. Keep responsibility for sign-off clear. If confidentiality rules, available time or the task’s stakes make a live trial unsuitable, clarify policy and practise with approved training or synthetic, non-sensitive material instead. For a hypothetical example, someone who routinely prepares an internal meeting summary might test an approved assistant on a non-sensitive, bounded draft, then check every factual point against the source notes. The example is a way to scope a trial, not a reported result or a claim that this task is suitable in every workplace.

Repeat the task only if the complete workflow meets its quality bar and the review burden is manageable; a quick first draft alone is not a reason to change practice. The free [AI task checker](/ai-job-risk-checker) can organize transparent task-level change-pressure signals. It does not provide a validated probability that you will lose your job. If you are weighing a broader move, the paid [career roadmap](/career-roadmap) compares staying and redesigning, an adjacent move, or a larger change against constraints such as salary floor, location, learning time and personal circumstances; it does not guarantee employment or income. Start with the permitted task you can evaluate and the standard its finished work must meet.

Questions readers ask