How to Test AI on a Recurring Task and Measure Review Time

What can a small workplace trial actually decide?

An AI draft can appear in seconds and still leave you with a slower task once checking, correction, and approval are finished. A small trial can answer a narrower question: should you continue this exact, permitted AI-assisted workflow, change one part, stop it, or collect a little more comparable evidence? It cannot establish that your team is more productive, that the occupation has become more exposed in practice, or that anyone’s job is at risk.

Treat the result as a local operational observation, tied to a defined task, tool configuration, reviewer, and period. Before starting, agree that the comparison concerns the same accepted outcome in both workflows. Record quality, total active effort, elapsed delivery time, permission to use the tool, and relevant spillovers as separate observations. That separation matters: a time difference does not cancel a quality failure or make an unapproved process acceptable, and a benefit to one worker may come with checking or correction work elsewhere.

NIST’s Artificial Intelligence Risk Management Framework (AI RMF 1.0) is voluntary organizational guidance that recommends context-relevant measurement, documenting results and uncertainty, and repeating assessment where appropriate. Those principles support transparency; NIST does not validate a personal stopwatch test, prescribe its case count, or turn it into a job-impact assessment. The employee-level protocol here is a proposed way to make a reversible workflow decision. Because a small set of cases can miss ordinary variation and cannot establish causal effects, its proper value is a decision for these conditions—not certainty beyond them.

Sources: Artificial Intelligence Risk Management Framework (AI RMF 1.0)

What is the task boundary, from trigger to accepted finish?

Write the boundary before anyone starts a timer. A useful template is: trigger; inputs ready at the start; steps included; accepted finish; and exclusions applied equally to both workflows. This makes the unit the work needed to deliver an accepted result, rather than the moment an AI tool produces something that looks complete. If a step exists only because AI was used—such as preparing a prompt, removing sensitive material, or checking a generated claim—it still belongs in the assisted path. If either workflow causes a correction downstream before the same recipient can use the result, include that correction too. Write the boundary in ordinary work terms, so another person can tell whether a case qualifies without guessing.

Consider a hypothetical internal weekly update assembled from meeting notes and project records. The trigger might be the Friday request for a status update; ready inputs might be the same set of approved notes and records. Included work could run from opening those materials through drafting, checking names and dates, resolving discrepancies, formatting the update, and sending it to the designated reviewer. The finish is the reviewer’s acceptance of an update that meets the agreed requirements—not the first draft, the first saved file, or simply pressing send.

For the ordinary workflow, list the steps actually used to produce that accepted update. For the AI-assisted workflow, list the same common steps plus AI-specific preparation, generation, verification, and repair. Keep the finish and exclusions symmetrical: if routine filing after acceptance is outside the boundary in one workflow, leave it outside the other. Do not omit a task merely because AI made it necessary. Conversely, unrelated work that would have happened anyway should not be charged to either condition just because it occurred during the same hour.

Case complexity can make a boundary too broad for a fair comparison. If some updates contain routine progress while others involve disputed dates, missing inputs, or sensitive decisions, define a narrower eligible variant—for example, updates with complete notes and no unresolved status changes. State that rule before selecting cases and retain a record when an apparent case falls outside it. The aim is to compare like enough work to interpret, not to quietly discard awkward cases after seeing which workflow they used.

A narrow boundary can still hide effects beyond the deliverable. Include changed work needed to reach the same accepted result, even if it occurs after drafting or with another reviewer. Separately note a spillover when AI shifts work to a colleague, changes the recipient’s follow-up, or delays another task; do not silently add unrelated work to the timed unit. The boundary should make the comparison readable while leaving visible any consequence that could change whether the workflow is worth continuing.

Which clocks answer different questions?

Use three clocks because “how fast?” can mean three different things. Time to usable first output records when there is something coherent enough to review. Total active human effort records the labor spent getting the deliverable through review and repair. Elapsed turnaround records how long the requester waits from the agreed start event to the agreed end event. Make active effort the main labor comparison; keep the other two as process diagnostics and a service measure.

For each case, use one row with: case identifier; workflow condition; start time; first usable output time; active minutes by step; pause or wait intervals; accepted completion time; and a short note on interruptions or unusual events. The start event should be a visible trigger, such as the moment the needed inputs are available and work begins. Record first usable output when you could reasonably hand it to a reviewer, not when a tool begins generating text. Stop the active-effort clock at the point the human work for that case is done, including any required correction; stop the turnaround clock when the agreed recipient-facing completion event occurs. Define these events before comparing cases and apply the same rules to both conditions.

Count active human minutes spent preparing materials, opening or arranging tools, writing prompts, reviewing output, checking facts, correcting errors, and reworking a rejected draft. Those steps can differ between workflows, but excluding them would hide labor simply because it happens before or after generation. If work is interrupted for an unrelated task, pause the active timer and note the interruption; resume when attention returns. If you cannot time continuously, make a quick contemporaneous estimate and mark it as estimated rather than silently treating it as precise. Use the same recording habit in both conditions so the method itself does not favor the workflow that feels more novel.

A tool or reviewer may leave a case waiting while the employee does other work. That interval belongs in elapsed turnaround, but not in active labor minutes unless the employee is actually monitoring, coordinating, or otherwise working on the case. If the employee switches to another task during an AI response, stop active time on the original case; do not count the overlap twice. A wait that blocks all further work can still matter to delivery, so preserve its start and end in the elapsed record. When a service commitment depends on responsiveness, report turnaround alongside labor effort instead of allowing one to stand in for the other.

At the end, summarize the cases with paired entries rather than a single blended number: first-output time, active human effort, and elapsed turnaround for each workflow. Then inspect where the difference appears. A quicker first draft followed by more checking may shorten one clock while increasing another. A long queue may delay delivery without adding employee labor. This log does not assign value to every minute or imply that the shortest elapsed interval is automatically preferable; it makes the tradeoff visible so the relevant decision can use the measure that matches its purpose.

What does acceptable quality mean before the clock starts?

Write the acceptance rule before timing so that both workflows face the same finish line. For a recurring internal update, the task owner might specify the required topics, agreement with the source notes, links or references needed to trace key claims, the agreed format, and whether the update is clear enough for its intended readers. Also name who can accept it. The person producing the update can check routine details, but a manager or subject-matter reviewer may own decisions that require authority or context the producer does not have.

A compact rubric can use three states without pretending to measure quality numerically. For example: “Pass” means every required topic appears, factual statements match the supplied notes, references are traceable where required, and the format and wording are fit for the intended audience. “Unmet” means a specified condition failed, such as a missing topic, unsupported statement, broken reference, or unusable format. “Unresolved” means the reviewer cannot decide from the agreed materials or reasonable criteria—for instance, two acceptable phrasings carry different emphasis and no preference was set. Record which condition applies and the reviewer’s reason; do not convert an unresolved judgment into a pass simply to finish the comparison.

Treat routine verification as part of completing the work, not automatically as a defect. Checking a figure against its source, confirming that a required topic is present, and making ordinary edits can be necessary in either workflow. A defect is a specific failure against the prewritten criteria; repair is the extra work needed to correct it. Record both the defect category and the active minutes used to repair it. This keeps normal review effort visible in the time measure while allowing the reader to see what kind of problems required additional correction.

A useful row can therefore keep separate fields for the three clock results, rubric state, defect type, repair minutes, reviewer and decision note. For a hypothetical update, one case might pass after ordinary source checks; another might be marked unmet because a required project change is absent; a third might remain unresolved because the source notes do not establish which of two conflicting dates is authoritative. These labels explain what happened. They are not scores to add together, and they do not make unlike defects equivalent. If the same kind of issue recurs, the log can show whether it is a source-input problem, a drafting omission, a traceability gap, or an unclear acceptance rule.

The distinction matters when reading the clocks. A draft delivered quickly but still marked unmet has not reached comparable completion; its correction work and eventual disposition should remain visible. A case that passes with more active effort may still meet the task, but the added effort is a real part of its labor result. A pass with no recorded defects does not prove that every statement was perfect; it means the named reviewer accepted it against the criteria that were actually agreed. If a reviewer changes the standard midstream, preserve the original criterion and record the change, then treat later cases under the revised rule as a distinct comparison.

Some work has no factual answer key and depends on editorial judgment, recipient needs, or professional discretion. In that situation, ask the accountable reviewer to state the criterion they will use before seeing which workflow produced the result, where practical. If reviewers disagree, retain the disagreement and its rationale rather than averaging opinions into a fictitious score. The study “Generative AI at Work” describes customer-support outcomes that include handling time, chats per hour, and successful resolution; that combination illustrates why completion and quality-related outcomes can be tracked alongside time. Its service setting does not supply acceptance criteria for an internal update. Those criteria must come from the task’s actual owner and intended use.

Sources: Generative AI at Work

How should comparable cases be chosen?

Choose the cases before you know which workflow will look better. First write a short eligibility rule for the recurring task: what must be ready, what ordinary variant is included, and what makes a case materially different. Then keep a dated list of every case that meets it. A worker need not create a formal experiment or randomize normal work to make a useful comparison. A predeclared sequence makes selection visible: for example, use the ordinary workflow for the next eligible weekly update, the approved AI-assisted workflow for the following eligible update, then alternate while the same configuration and acceptance rule remain in place. If alternation would disrupt a deadline or violate policy, record the practical reason and use the next eligible case rather than quietly choosing the most favorable one.

Pairing can make differences easier to inspect when two cases resemble each other. A pair might contain updates with similarly complete inputs, a similar number of source documents, the same reviewer path, and roughly the same kind of requested change. Record those features beside each case, not as a score that claims to erase complexity. For instance, note whether notes were complete, how many sources were checked, whether the reviewer was the same, and whether the work involved a routine status update or a disputed change. When no close pair exists, retain the case and mark the difference. A smaller set of imperfectly matched cases is still informative as a description of what happened, provided the mismatch stays visible in the interpretation.

Keep every eligible case in the log, including exceptions, and state why its workflow differed or why it could not be completed. A difficult case should not disappear because it took longer; nor should an AI-assisted case be forced into an ordinary category after the tool fails or is unavailable. If the task cannot proceed under the planned condition, record the interruption and disposition separately. This protects against a common selection problem: choosing easy cases for one workflow and complicated ones for the other can create an apparent time difference that belongs to the case mix. It also avoids the reverse error of using a few unusually messy assisted cases to dismiss a workflow that was not tested on the ordinary work it was meant to support.

The METR article “We are Changing our Developer Productivity Experiment Design” describes selection and measurement problems in a later study of developers using coding agents. The update reports that some developers sometimes withheld tasks they did not want to do without AI, alongside concerns about task substitution and concurrent work during agent waits; its authors judged that study’s estimate unreliable. That account motivates recording eligibility, exclusions, and workflow changes. It does not establish that an employee’s own trial is biased, and it is not the result of METR’s earlier randomized trial. Keep the source attached to this methods warning rather than using it to reinterpret the earlier result.

Familiarity also changes during a trial. The employee may learn a better prompt, become quicker at finding errors, or simply become more practiced at the underlying task. Record whether the workflow, tool, model/version, settings, or training changed, and note practice effects that could plausibly matter. If a configuration changes substantially, label the later cases as a new run instead of blending them into one comparison. Alternating can distribute gradual practice across conditions, but it cannot remove learning, changing inputs, reviewer variation, deadlines, or other differences. The point is to make those influences inspectable, not to claim a clean causal estimate.

A short convenience sample cannot support a causal percentage such as “AI cut review time by 20%.” The cases were not necessarily representative, matched on every relevant feature, or numerous enough to separate workflow effects from ordinary variation. Report the observed cases and their context in plain language: for these eligible updates, under this tool setup and reviewer path, active effort was usually lower, higher, mixed, or too inconsistent to interpret. That is a practical judgment about whether to retain or adjust this workflow. If a high-stakes policy or staffing decision depends on a causal estimate, an employee log is not a substitute for a properly designed evaluation.

Sources: We are Changing our Developer Productivity Experiment Design

Why do impressions of speed mislead?

Measure the time to accepted completion, including review, because the visible speed of generation is only one part of the task. An AI assistant can produce a draft quickly while leaving a person to check its claims, compare sources, repair omissions, and obtain acceptance. Those later steps are less concentrated than watching text appear: they may be spread across several pauses and small edits. If the worker remembers the striking generation moment more readily than these dispersed tasks, the overall workflow can feel faster even when the work unit takes longer. That is a plausible measurement mechanism for an individual trial, not a claim that every worker misjudges every AI task.

The randomized study “Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity” offers a bounded example of this gap. It studied 16 developers with moderate AI experience completing 246 tasks in mature open-source repositories. In that setting, the reported average task-completion time was 19% longer when AI tools were allowed. Before the tasks, participants expected AI to reduce time; after using it, they still estimated that it had reduced time. The contrast is useful because the researchers recorded task completion while participants’ impressions pointed in the opposite direction. It shows why perceived speed and measured completion should be recorded as different things in a workflow test.

The finding’s unit matters. These were experienced developers working on tasks in mature software repositories, using tools available during the study period in 2025. The result does not estimate what an employee in another occupation will experience, how much review time a particular task requires, or whether AI has no value. Nor does an average completion-time result tell a task owner whether the work met the required quality bar; that needs the previously defined acceptance decision. Treat the experiment as evidence that intuition can diverge from a measured outcome in a particular setting, not as a transferable slowdown factor to apply to your own log.

The practical implication is to define a finish event that includes the work needed for an accepted result, then capture active effort across the entire path. A first draft is a useful process milestone, but it is not equivalent to completion if someone must verify sources, correct a figure, restore omitted context, or wait for a reviewer’s decision. If an assistant appears to save ten minutes at the start but adds brief checks throughout the task, a person relying on the salient first stage may not mentally total those later minutes. A contemporaneous record makes the less visible work count without asking memory to reconstruct it at the end of the day.

Self-estimates can still be recorded as a separate observation: after a case, note whether the workflow felt quicker, slower, or about the same before looking at the accumulated timings. Comparing that impression with the log can show whether this worker’s sense of speed tracks the measured completion pattern for this task. Do not turn a mismatch into a personal failing. Perception can be shaped by interruptions, the novelty of a tool, uncertainty about what counts as work, or a genuine benefit that the stopwatch does not capture. The useful response is to inspect which stage produced the difference and whether the task’s purpose was fulfilled, rather than assume that the feeling or the clock alone settles the decision.

The METR study should also be read in time: it evaluated early-2025 tools and a specialized repository task setting. Assistants, workflows, and skill with them can change, and a different task may place far less or more weight on checking. Its result is therefore a reason to measure locally, not a forecast for current tools or a universal estimate of AI’s effect. For this employee’s question, the informative comparison is whether the permitted configuration repeatedly reaches the same accepted outcome with a different amount of active effort and turnaround under the conditions actually logged.

This distinction matters even when a generated draft is visibly impressive. Fluency is immediate; verification often consists of quiet acts whose value is preventing a wrong, incomplete, or unusable result. If those acts are excluded because they do not look like drafting, the measured workflow rewards speed at producing material rather than speed at completing the task. Conversely, if the assisted path truly reduces checking because relevant facts are traceable and acceptance is straightforward, the log can show that too. Recording review does not presume that AI creates extra burden; it ensures both a burden and a reduction in burden can appear in the observed outcome.

A worker should therefore compare impressions with a defined, recorded completion measure rather than borrowing the METR percentage. In the study, participant expectations and estimates did not match recorded average time for the sampled developer tasks. In a local trial, the same kind of divergence could arise in either direction, and a few cases may remain mixed. The defensible takeaway is methodological: measure through acceptance and retain the review work in the account. Whether that changes this employee’s decision depends on the actual cases, the quality rule, and the work context—not on an inference from developers’ repository tasks to a different job.

Sources: Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity

How can case selection and waiting distort a comparison?

Even after a workflow is assigned, the comparison can become unbalanced if the cases that reach completion differ from those originally eligible. An AI-disallowed case may be postponed, declined, or replaced because the worker expects it to be difficult without assistance. Conversely, a case that proves awkward with AI may be quietly moved into the ordinary workflow. The final timed set then reflects both workflow and decisions about which work to attempt. A neat table of completed cases cannot reveal that selection by itself; the missing cases matter too.

METR’s “We are Changing our Developer Productivity Experiment Design” reports this kind of concern in its later developer study. The update describes participant feedback that some developers sometimes withheld tasks they did not want to do without AI, as well as concerns about task substitution and quality differences. The authors say these issues make the later study’s current effect estimates unreliable. This is a warning about how that study unfolded and how its estimate should be read. It is separate from the result of METR’s earlier randomized trial, and it does not revise that trial into a different finding.

For a workplace test, make the denominator visible before workflow choice can influence it. Keep a dated list of eligible recurring cases, including cases later declined, postponed, interrupted, or excluded. For each, record whether it was attempted and, if not, the reason: policy, unavailable inputs, time pressure, tool failure, or a decision not to proceed. These entries need not be elaborate. Their purpose is to show whether one condition systematically loses the cases that would be most informative. If a case cannot safely or reasonably be attempted, preserving that fact is more honest than treating the absence as a neutral result.

Task substitution creates a second interpretive problem. If a planned report is replaced by a shorter summary when AI is available, the resulting times refer to different work. The assisted condition may appear faster because it delivered a narrower output, while the human-only condition still met the original request. Record any change in deliverable, scope, or completion standard and do not compare those observations as if only the tool differed. When a revised task is itself useful, it can be evaluated as a separate workflow with its own acceptance condition; it should not silently stand in for the original task.

Waiting can distort the labor measure even when the requested output stays constant. An agent may run for several minutes while the employee handles unrelated work. Those minutes increase elapsed turnaround if the case remains open, but they do not all represent active labor on that case. If the worker monitors progress, resolves a tool issue, or prepares the next step, record that active effort separately. If another task proceeds independently, preserve the overlap in the timeline and avoid charging the same minutes twice. This distinction matters because parallel work can make an agent workflow look productive in elapsed time while leaving uncertain how much attention or coordination it required.

A compact log can therefore record each case’s eligibility, attempted condition, exclusion or substitution reason, configuration, active work intervals, agent wait intervals, unrelated work during waits, and elapsed completion. If the tool, model, settings, or task changes materially, mark a new run rather than merging results into one average. This is especially useful when the changed setup alters how long the agent waits or what work the employee does during that interval. The record will not correct selection after the fact, but it makes the changed comparison explicit and helps locate the source of an apparent difference.

The METR update concerns a developer study involving coding agents; it does not establish that all employee trials are biased or that waiting always creates a misleading result. It identifies conditions worth checking when they also occur locally: cases disappear selectively, the assigned task changes, or AI wait time overlaps other work. If none occurs, the warning may have little bearing on the trial. If one does occur, interpret the observed times as outcomes of the actual mix of cases and work patterns, not as a clean estimate of what the tool alone caused.

Sources: We are Changing our Developer Productivity Experiment Design

What do workplace field experiments add—and leave open?

Workplace field experiments add evidence that AI effects can be studied while people perform work inside actual employer settings. The paper “The Effects of Generative AI on High-Skilled Work: Evidence from Three Field Experiments with Software Developers” reports randomized field experiments with software developers at Microsoft, Accenture, and an anonymous Fortune 100 company. That establishes a relevant kind of evidence: a tool can be evaluated in organizational contexts where employees have real work, workflows, and workplace arrangements. It gives a stronger basis for asking about deployed use than a demonstration that a model can complete an isolated task.

The paper’s full text is access-restricted in the available record, so the claim here is limited to its official abstract and bibliographic page. Those materials support the study design and setting just described. They do not support adding detailed methods, effect magnitudes, or further conclusions about how review time was counted. The reported experiments concern software developers and coding-task productivity; they do not supply a general rule for measuring quality-adjusted time in every role. Keeping that boundary visible prevents a real-world study label from being mistaken for evidence about outcomes the accessible source does not establish.

A workplace experiment and an employee’s recurring-task log answer related but different questions. A field study can compare outcomes under a defined tool and employer arrangement across participants or work units. A local log asks whether one employee, doing one recurring task under the rules and review path available to them, sees a useful change. The local unit may be a weekly update, a client response, a requirements summary, or another repeated deliverable. Its inputs, acceptance decision, and interruptions can be unlike coding work even when both involve drafting or synthesis.

Transfer depends on the chain from setting to outcome. The tool’s capabilities and permitted configuration affect what work it can perform. The task determines how much of the output is routine, context-dependent, or costly to verify. The employer’s workflow determines when the tool is used, who checks its output, and whether an AI-produced draft changes handoffs or waiting. The outcome measure determines whether a reported productivity change concerns completion time, throughput, quality, or some combination. A finding can be sound within its study and still leave these features unresolved for a different employee’s task.

The organizational setting is part of the treatment being observed, not background decoration. A field experiment may operate with training, access controls, approved tools, task routing, or review expectations that an individual worker does not control. Another organization might use the same tool with different defaults, documentation, security rules, or accountability. If a local employee cannot use the same system or has a different approval burden, copying the study’s headline would erase a condition that could shape the result. The useful comparison is to identify which of those arrangements match and which would need to be tested locally.

This does not make field evidence irrelevant. It helps generate a plausible hypothesis about a mechanism worth checking, such as whether assistance changes drafting throughput or shifts effort into review. The hypothesis should be translated into the local task’s own finish line and measured outcomes. If the field study assesses coding-task completion, an employee writing internal updates should not presume that its unit is equivalent to accepted completion of those updates. They can instead ask what work is done before acceptance, what review is required, and whether an approved AI workflow changes those steps.

A practical transfer note can stay brief: name the study’s population and work setting, the AI arrangement it examined as far as the source verifies, the outcome it reports, and the local difference most likely to matter. For this paper, the verified setting is software developers in three firms and the stated focus is coding-task productivity; detailed operational comparisons should wait for accessible methods. For a local trial, write down the employee’s task variant, tool and settings, review path, and completion rule. This makes the jump from research to workplace observation explicit rather than implicit.

Multiple firms improve the range of employer contexts represented compared with a single-site study, but three software firms still do not represent every occupation, organization, or reviewer arrangement. Nor does a field experiment automatically reveal what happens when a worker’s own eligible cases are skipped, substituted, or completed while an agent waits. Use workplace studies as evidence that real-use effects can be investigated and as sources of questions for a local test. Let the employee’s own recorded cases answer the narrower decision: whether this permitted workflow reaches the required result with a different pattern of effort and elapsed time.

Sources: The Effects of Generative AI on High-Skilled Work: Evidence from Three Field Experiments with Software Developers

Why keep quality and completion separate from minutes?

Because minutes describe resource use, while quality and completion describe whether the work did its job. A workflow can produce an accepted result with less active effort, produce a faster but unusable draft, or take similar effort while improving the result for its recipient. These are different observations. Record each against the same task boundary and acceptance criteria; do not make a fast first output stand in for finished work or assume that a longer process is automatically better.

The field study “Generative AI at Work” makes this multi-outcome view concrete in a customer-support setting. Its accessible abstract describes an AI assistant deployed in customer support and identifies several operational productivity components: time per chat, chats handled per hour, and the share of chats successfully resolved. The study therefore treats speed, throughput, and successful resolution as distinguishable outcomes rather than interchangeable names for one stopwatch reading. The abstract also mentions worker experience, another dimension that cannot simply be translated into minutes. This is a useful measurement example: service work can change along more than one axis, and a single metric may miss a relevant part of the result.

That study does not supply a ready-made scorecard for an employee writing reports, preparing analyses, or answering internal requests. Customer support has its own interaction flow and definition of resolution; another task owner must specify what counts as accepted work for the task at hand. The accessible record also does not provide a basis here for quoting effect sizes. Use the study to motivate separate observations, not to transfer a result or prescribe a quality rubric. The task’s accountable reviewer and recipient determine which quality requirements matter.

A compact record can put the outcomes side by side: accepted quality—met, unmet, or unresolved against the agreed criteria; active effort—lower, similar, higher, or mixed across observed cases; turnaround—earlier, similar, later, or mixed at delivery; and separately observed benefit or cost—what changed for the recipient or workflow, and how it was noticed. For each entry, identify the case and conditions. This is a decision table, not a composite score: do not add categories together or let an improvement in one erase a failure in another.

A time saving can still matter when quality does not improve, provided the task’s purpose remains fulfilled. If an accepted response takes less active effort and arrives in time, the worker may reasonably value that efficiency on its own. Conversely, a perceived benefit such as clearer handoff, fewer follow-up questions, or easier access to a usable draft should be named and observed separately; do not claim it from faster handling alone. If the task is consequential, the relevant owner may decide that one quality failure outweighs several quicker cases. The record makes that trade-off visible without pretending every dimension has a common unit.

Read the pattern conditionally. If acceptance holds and effort falls while turnaround stays suitable, the workflow may be useful on efficiency grounds. If speed improves but acceptance fails, the result is not successful completion under that task’s criteria. If accepted quality improves without a time saving, the gain may still matter when the recipient or owner values it and the added effort is acceptable. Mixed results call for inspecting which case feature or workflow step differed, rather than averaging away the distinction. The judgment belongs to the task and its purpose, not to an abstract productivity label.

Sources: Generative AI at Work

When should the employee continue, adjust, stop, or call it inconclusive?

Use the observations to choose the next reversible workflow decision, not to award a winner from one attractive timing. First ask whether the tested configuration was permitted and whether cases met the task’s acceptance conditions. Then consider whether total active effort changed consistently enough to matter for this recurring work, whether turnaround remained fit for purpose, and whether any separate benefit or cost was actually observed. Keep these questions distinct: a favorable time pattern cannot cure a policy breach or an unacceptable result.

Continue when the approved configuration repeatedly produces work that meets the agreed criteria, the review and accountability path remains workable, and the observed effort or other valued outcome supports keeping it for this task. Continue means retaining the tested arrangement for now, with its current tool, settings, inputs, and review process. It does not mean declaring the tool effective in general. If the advantage is mainly faster completion, say so; if the reason is a documented non-time benefit, name that benefit and who values it. Do not silently convert either finding into a broader claim about the employee’s role.

Adjust when the record points to one identifiable friction that can be changed safely: perhaps a prompt routinely omits a required field, source material arrives in inconsistent form, or review repeatedly catches the same correctable error. Change one relevant part of the workflow, preserve the acceptance standard, and treat materially different settings as a distinct configuration in the log. This makes the next observation interpretable: did the adjustment address the named friction while preserving accepted completion? Avoid changing the tool, task scope, reviewer, and finish criteria together, because then a better or worse outcome has no clear explanation.

Stop or suspend the AI-assisted path when it produces a material quality failure, breaches policy, exposes information that should not be entered, obscures who is accountable, or leaves the reviewer unable to verify the result. A safe fallback is more important than completing a timing comparison. Record what occurred and the disposition, then involve the person responsible for the task or tool approval when needed. Stopping this configuration is a local operational decision; it does not establish that every possible AI workflow is unsuitable. A revised setup would need permission and its own reviewable conditions before use.

Call the result inconclusive when cases point in different directions, a change in inputs or review conditions prevents a fair reading, or too little relevant recurring work has occurred to see a pattern that would affect the decision. State what is uncertain rather than compressing mixed evidence into an average verdict. A realistic next observation might be the next comparable case with complete inputs, or the same workflow after one clearly logged adjustment. More cases are useful only when the work recurs, comparisons remain meaningful, and the decision can still be changed; there is no universal minimum case count or confidence threshold for an employee’s practical log.

Waiting for more observations also has a cost. If the task is rare, the potential harm is serious, the policy is unclear, or no accountable reviewer has agreed to the acceptance conditions, pause and seek a decision rather than extending an informal trial. If the work is routine and low consequence, a small number of further comparable cases may be enough to decide whether to retain or refine the tested workflow, while still leaving uncertainty. The point is proportionality: gather more information only if it can realistically change a reversible choice and doing so is acceptable under the task’s risk and rules.

Keep non-time outcomes visible through the decision. If accepted quality holds and active effort falls, continued use may be justified by saved effort even without an added quality benefit. If quality or recipient value improves while effort does not fall, decide whether that specific benefit warrants the additional work; do not label it a speed gain. If the benefit is merely expected but was not observed, keep it as a hypothesis for a later check, not a result. The final choice should name its basis—accepted completion, effort, turnaround, a separate observed outcome, or a constraint—and the condition that would prompt review.

What can this result say about the wider role?

A result from one recurring task can describe that workflow under the conditions observed; it cannot, by itself, describe how much of a whole role is exposed or likely to change. The ILO’s “Generative AI and jobs: a refined global index of occupational exposure” works at the occupation level, using task judgments to characterize potential exposure. The OECD’s “Beyond automation: Decoding the impact of Generative AI on regional labour markets” models a different broad unit: the regional share of job tasks that could be done in half the time with generative AI. Both help frame where potential applicability may be concentrated across occupations or regions. Neither tells an employee how many minutes their own review took, whether their employer has adopted a tool, or whether a role will be redesigned.

Those measures sit at different scales and answer different questions. An occupational index summarizes task-based exposure across a job classification; a regional model estimates potential applicability across local labor markets. The employee’s log instead records actual cases of one bounded task, including its output and review path. It can show that this variant required less or more active effort in the tested setting, subject to the limits of those observations. It cannot establish the share of the occupation represented by that task unless the employee maps the rest of the work separately. And neither broad exposure evidence nor one local timing result establishes personal displacement: adoption, workflow redesign, demand, and organizational choices remain additional steps between technical applicability and a worker’s outcome.

A useful next move is to place the tested task in a descriptive role inventory. For that task, note how often it occurs, what output it produces, which inputs or relationships it depends on, who reviews and accepts it, and whether an approved tool is available. Then consider other recurring tasks separately, using the same plain descriptors rather than extending the first result to the entire job. Tasks that look similar on a title may differ in source access, exceptions, accountability, or the cost of checking an output. A task inventory also makes omissions visible: irregular escalations, coordination, and follow-through may matter even when they are hard to time as standalone tasks. This inventory is a way to see where the local observation belongs; it is not a validated assessment or a calculation of personal job-loss odds.

Scale matters when interpreting a favorable local result. If the task is infrequent, its time effect may have little bearing on the role’s overall work mix. If no adoption is planned, a technically applicable task may remain unchanged in the employee’s actual workflow. Those conditions are reasons to record frequency and organizational context alongside the task result, rather than to inflate a narrow gain into a career forecast or dismiss it as irrelevant. The log can inform a conversation about this workflow; occupational and regional evidence can supply context for questions about the surrounding work, while leaving the employee’s role-level conclusion open until its other tasks and adoption conditions are considered.

Sources: Generative AI and jobs: a refined global index of occupational exposure; Beyond automation: Decoding the impact of Generative AI on regional labour markets

What conversation should happen next?

Ask the manager, task owner, or reviewer: “Could we agree on one permitted version of this recurring task, what counts as accepted, and who will review the next case?” Before using AI, settle three points: the approved tool and inputs; the finish criteria and accountable reviewer; and a brief log of the case, active effort, elapsed turnaround, and any correction. Also agree what happens if required source material is missing or the output fails review, so the employee knows when to fall back to the existing process. If access or quality authority sits elsewhere, ask for approval and reviewer-defined conditions first. That resolves the practical next step without treating this small trial as a verdict on the job. The next comparable case can then be recorded against an agreed finish line, including whether it was accepted and what review or repair it required.

For broader context, the free checker at /ai-job-risk-checker offers task-level change-pressure signals, not a validated probability of displacement. If you want to compare larger career paths against your constraints, the paid roadmap at /career-roadmap is a separate option; it does not guarantee employment or income. Start with the approved next case and the agreed review.

Questions readers ask