What Should Policy Analysts Verify in AI Research Summaries?

What does NIST actually ask organizations to verify?

NIST’s Generative Artificial Intelligence Profile gives organizations two particularly relevant actions when generated research summaries are involved. Under MEASURE 2.5, action MS-2.5-003 says to review and verify sources and citations in generative AI system outputs during pre-deployment risk measurement and ongoing monitoring. Under MAP 2.3, action MP-2.3-003 says to deploy and document fact-checking techniques for the accuracy and veracity of generated information, with special attention when information comes from multiple or unknown sources. In practical terms, the Profile asks organizations to assess whether their systems produce traceable, supportable information and to make fact-checking part of risk management. It does not set out a personal checklist for policy analysts reviewing a single summary. That distinction matters because the Profile is voluntary, cross-sector guidance for organizations. It describes itself as an implementation of the AI Risk Management Framework for generative AI, intended to help organizations manage risks across system design, development, use, and evaluation. Its suggested actions address systems, processes, and organizational roles; they are not a rule that every analyst must follow in a fixed order. The NIST AI Risk Management Framework: Generative Artificial Intelligence Profile also identifies confabulation as confidently presented erroneous or false content, output that diverges from the prompt or other input, or statements that contradict one another within the same context. It warns that generated citations can appear to justify an answer and may mislead people into trusting it. That warning identifies a way misplaced trust can arise. It does not establish that every citation is false, or provide an error rate for research summaries. For a policy analyst, the useful next step is an editorial application of those system-level recommendations: treat the summary as a set of claims that need checking before they carry analysis. Open the cited material and see whether it exists and is the intended source; find the relevant evidence; compare the summary’s wording with what that evidence actually says; and note any qualification or gap that matters to the decision. This translates NIST’s source-verification and fact-checking actions into a review of one work artifact, but the ordered sequence is not a NIST-prescribed or validated protocol. A citation that opens successfully is only a starting point: it does not by itself show that the cited passage supports the sentence, that the source is reliable or current, or that other important evidence has been included. The analyst must keep those questions distinct and decide what the summary will be used for before deciding how much checking is enough. Attribute the system-level recommendations to NIST; describe the claim-by-claim review as a practical application.

Sources: Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile | NIST; AI Risk Management Framework | NIST

How do you test whether a claim survives contact with its source?

For each consequential sentence in a research summary, follow its citation to the cited document and then to the exact passage, table, or data offered as support. Ask five questions in order: Is this the intended source, can I access it, where is the relevant evidence, does that evidence support these exact words, and have the sentence kept the source’s scope and qualifications? A working link is only the start of verification. This claim-to-source sequence is an editorial way to apply NIST’s recommendations, not a checklist prescribed by NIST for policy analysts. In the Generative AI Profile, NIST calls for reviewing and verifying sources and citations in system outputs as part of measurement and monitoring, and for documenting fact-checking techniques, especially when information comes from multiple or unknown sources. That guidance supports checking the citation trail. This sequence is an editorial application, not NIST’s protocol. Begin with source identity. Does the citation lead to the study, official record, dataset, or policy document the sentence names? Is it the final version or a draft, and does it concern the same jurisdiction, program, or time period? If the cited material cannot be found, mark the claim unresolved rather than treating the citation text as proof. Next locate the evidence inside the source. A report may contain many findings, while only one table, paragraph, or note bears on the summary’s sentence. Find that specific support and read enough surrounding context to understand the measure and its limits. If the summary says a program reduced costs, for example, identify where the source reports that result and what comparison it actually made. Then test semantic fit: would a careful reader reasonably paraphrase the located evidence in the same way? Imagine a policy summary describing a proposal as an adopted policy. The document may discuss the proposal, but that does not support the stronger status claim. Or imagine a study reporting that two measures moved together, while the summary says one caused the other. A citation can be genuine and still fail to support the wording. Revise the sentence or find stronger support. Finally check scope and qualifications before deciding what to do with the sentence. Imagine a study of a bounded sample summarized as applying to an entire population, or a figure presented without the reporting period that gives it meaning. Compare the summary with the source’s population, definitions, dates, method, and stated limitations. Keep material qualifications close to the claim. If the source supports only a narrower statement, narrow the wording; if another source is needed, add it; if support remains uncertain, label the gap. The disposition can be simple: retain a supported sentence, narrow an overstated one, cite a better or additional source, or mark it unresolved. This pass can catch a wrong, inaccessible, irrelevant, or weakly matched citation, and wording stronger than its evidence. It cannot, by itself, establish that the underlying source is credible or up to date, or that the chosen evidence is sufficient. A clean citation trail is necessary, but does not guarantee that the claim or summary is sound.

Sources: Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile | NIST

What changes when the wording outruns the evidence?

Check whether the summary preserves the source’s scope and strength: who or what was studied, when, under which definitions and method, and with what limits. If the source supports a narrower statement, revise the sentence or state that the claim is not established. A citation can lead to a genuine document and still fail to support the wording attached to it. The question is whether its evidence warrants this sentence at this level of certainty. Compression can alter a finding. Imagine a report describes an early pilot in a limited setting. A summary saying the policy “works” turns an initial test into a settled conclusion. If a government document proposes a measure, saying it “is current policy” changes its status. If a study reports that two outcomes occurred together, saying one “caused” the other adds a causal conclusion. A local result written as a population-wide rule, or an estimate without its uncertainty presented as settled, makes a similar leap. These are hypothetical examples of scope changes, not errors NIST reports testing in policy summaries. The relevant NIST concept is confabulation. In *Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile* (NIST AI 600-1), NIST describes confidently stated erroneous or false content, as well as output that diverges from its prompt or other input or contradicts earlier statements in the same context. It warns that fabricated citations or reasoning can make an answer seem more trustworthy. Polished wording and a plausible reference therefore do not settle whether a summary faithfully represents its source. This definition does not establish an error rate for policy summaries or mean every generated summary has a problem. Compare the summary’s sentence with the source’s own language and boundaries. Ask what population, place, period, or document the source covers; whether key terms mean the same thing in both texts; what method produced the result; and which qualifications accompany it. Then compare the verb and certainty. “Observed,” “estimated,” “proposed,” and “may” carry different claims from “proved,” “adopted,” or “will.” If a summary drops a qualification that changes the conclusion, restore it close to the claim rather than leaving a broad sentence followed by a distant caveat. Keep another boundary visible: what the source reports and what the analyst infers are separate claims. A document may accurately report a finding, while the summary adds that the finding means a policy should be expanded. That recommendation may be reasonable, but it depends on further premises about goals, trade-offs, costs, or applicability. The source’s support for its result does not automatically support that reasoning. Mark the transition as analysis and avoid presenting the inference as if it appeared in the source. When wording exceeds evidence, narrow the sentence to the population, period, status, or relationship the source supports. Restore a material qualification. If the document does not establish the claim, remove it or say it is not established by the source reviewed. If the source is ambiguous, preserve that uncertainty instead of choosing the more decisive reading. The aim is not to weaken every sentence; each should be no broader or more certain than its evidence. This matters when a summary passes to another analyst, manager, or decision document. Readers may see the compressed sentence without reopening every source, so qualifiers need to travel with the claim. A concise record can state the finding, its boundary, and any separate inference, helping the next reader see where evidence ends and judgment begins. A verified citation is a starting point; faithful wording lets it support the sentence actually being used.

Sources: Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile | NIST

How do you tell a sourced summary from a complete synthesis?

A source check asks whether a document supports a sentence. A synthesis check asks whether selected documents fairly support the summary’s conclusion. A summary can cite accurately yet omit a counterpoint, rely on a narrow slice of evidence, or imply agreement the sources do not establish. NIST’s Generative AI Profile recommends verifying sources and citations and documenting fact-checking techniques, especially for information from multiple or unknown sources. It does not define a completeness test for policy summaries; the distinction here applies that guidance to an analyst’s task. Start with the question and its boundaries: what decision should the summary illuminate, and which jurisdiction, population, period, and policy status does it cover? They determine relevance. A proposal should not silently become current policy, and one jurisdiction’s document cannot establish what is happening elsewhere. Inspect whether the source set includes material needed to answer the question, such as policy documents and evidence about effects. Are dates and statuses visible? Does it distinguish primary documents from commentary? If it claims consensus, ask what was searched, included, and possibly missed. Citations alone cannot establish breadth. If the source set is not traceable, treat the text as a starting point for finding documents, not a complete review. Look for disagreement and qualifications that could change the reading. One source may describe a policy’s aim without evaluating outcomes; another may report an outcome only for a bounded group or period. A concise summary need not reproduce every detail, but should not erase a limitation or contrary result that materially weakens its conclusion. If an omission changes the claim’s direction, reach, or certainty, include the qualification or narrow the conclusion. Accurate accounts of individual documents do not by themselves make a balanced synthesis. Evidence-synthesis research cautions against treating a summary as proof of coverage, though its scope matters. The peer-reviewed “Generative artificial intelligence use in evidence synthesis: A systematic review” searched five databases through January 17, 2025, and included 19 comparative studies. In three literature-search studies, reported recall ranged from 4% to 32%. Many included studies had high or unclear risk of bias in study selection, system conduct, or applicability. The review concluded that evidence did not support synthesis without human oversight, while noting a possible assisting role for many tasks. This research concerns evidence-synthesis tasks, not policy analysts checking every kind of summary; its evidence base was small and heterogeneous, with health settings prominent. The figures do not estimate omissions in policy summaries, but show why a polished result cannot certify its search was complete. If sources and search path are visible, compare them with the question, check for disagreement, and state the boundary. If sources are missing, inaccessible, or selected through an unknown process, citations that happen to be present do not establish coverage. Search explicitly, request documents, or narrow the claim to material checked. Under deadline pressure, disclose the limit rather than calling an unexamined source set comprehensive. Completeness is relative to purpose, not exhaustive reading: have a defensible basis for the claimed scope and do not hide evidence that would materially alter interpretation. A citation pass is necessary before a summary informs analysis, but cannot alone establish representativeness or balance. If the evidence base cannot be traced, keep the summary as a reading aid. Call it a synthesis only when its source set, boundaries, and important qualifications can be examined.

Sources: Generative artificial intelligence use in evidence synthesis: A systematic review; Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile | NIST

What can AI-assisted research save, and what must the analyst still own?

AI assistance may speed some evidence-review tasks, but the available comparisons do not show that every generated summary is accurate, complete, or faster to produce. Treat it as a draft or navigation aid; a person still needs to check the claims, the evidence gathered, and the reasoning that connects them. A useful comparison is not a contest between a machine and an analyst. It is a comparison between bounded tasks that may be accelerated and the evidential decisions that remain accountable to the analyst.

The UK government publication AI-assisted vs human-only evidence review reports an exercise commissioned by the Department for Science, Innovation and Technology and the Department for Digital, Culture, Media and Sport. Two researchers separately reviewed the same topic, “How technology diffusion impacts UK growth and productivity,” using the same brief and inclusion criteria. One review used human-only methods; the other combined generative AI tools with manual checks and edits. The assisted review took 23% less time overall, and its analysis phase took 56% less time. Those figures describe this particular exercise, not an expected saving for a policy team. The initial AI-assisted draft was judged less fluent and needed more revisions. The publication also says the tools produced errors that required manual verification and explicitly describes the exercise as a case study whose results are not generalisable. It offers a credible example of acceleration in selected work, alongside review and correction costs, rather than proof of a universal productivity gain.

A broader but still limited comparison appears in Generative artificial intelligence use in evidence synthesis: A systematic review. It included 19 peer-reviewed comparative studies of evidence-synthesis tasks, such as searching, screening, data extraction, and risk-of-bias assessment. The review found variation by task. For example, three search studies reported recall between 4% and 32%, meaning many relevant records found by human searchers were not found in those evaluations. Across other tasks, errors also varied; the review reports that most included studies had high or unclear risk of bias in at least some domains, often because of small or restricted samples, convenience selection, or evaluation choices. Most studies came from health settings, with two in information technology. The evidence therefore warns against assuming an AI-produced source set is complete, but it does not measure the accuracy of every policy summary or current system. The authors conclude that expert human oversight remains necessary for the tasks they reviewed.

A Primer for Evaluating Large Language Models in Social-Science Research adds a related methodological caution, not a workplace accuracy estimate: evaluation depends on transparent methods and materials, and results can be difficult to reproduce as systems and access change. Taken together, the sources support a measured position. AI may organize material or accelerate a defined synthesis step; the analyst must still decide which sources belong, whether evidence supports the wording, what context matters, and whether the conclusion follows. Observed performance in these research exercises does not establish adoption across employers, productivity at scale, occupation-wide exposure, or displacement. Keep the tool's contribution visible in the workflow, verify decision-relevant claims against their sources, and retain responsibility for the final evidential judgment.

Sources: AI-assisted vs human-only evidence review; Generative artificial intelligence use in evidence synthesis: A systematic review; A Primer for Evaluating Large Language Models in Social-Science Research

How should the intended use change the depth of review?

How much review is enough depends on what the summary will do next. NIST’s AI Risk Management Framework (AI RMF) Core asks organizations to understand intended purpose and context, document system knowledge limits and how people may use and oversee outputs, and define oversight in line with organizational policies. Those are system risk-management outcomes, not a personal review scale for policy analysts. The Core also says its actions are not a checklist or necessarily an ordered sequence. The levels below are an editorial decision rule drawn from that context principle, not a NIST-mandated tier system. If the summary is only an orientation aid to help choose which documents to read, a lighter check can be proportionate: label it as an unverified guide, inspect enough of its references to find the relevant material, and keep it out of the evidentiary record until checked. The key condition is that a reader will not mistake the digest for a finished account of what the sources establish. A visible gap is tolerable when the next step is to open the documents; it is not tolerable if the digest will be forwarded as the finding itself. When the summary becomes an input to analysis, verify the claims that carry the reasoning. Follow their citations to the source passages, then check that the summary has not changed the population, date, definition, or qualification in a way that would alter the interpretation. This does not require rechecking every sentence with equal effort. It means identifying which claims the analysis depends on and checking those before building on them. NIST’s AI RMF Core connects system context and intended use to how outputs may be used and overseen; applying that idea here means the more a summary shapes the analyst’s conclusion, the more its supporting evidence needs to be visible. If the summary supports a policy recommendation, review must also address the evidence limits or disagreements that could change the decision. A citation check can show that a source supports one sentence; it cannot by itself show that the selected sources represent the relevant evidence or that omitted qualifications do not matter. The analyst should be able to explain what is established, what is an interpretation, and what remains uncertain. Then follow the organization’s review and sign-off rules, since the AI RMF Core places oversight processes within organizational policy and calls for clear roles and responsibilities. NIST does not set a universal sign-off level for a policy memo. This is a consequence-and-reversibility judgment, not a word-count rule. A short, fluent summary can still contain a controlling unsupported claim; a longer digest may be useful for navigation while remaining unsuitable as evidence. If time is limited and a key source cannot be checked, time pressure does not convert the claim into verified support. Narrow the analysis, qualify the recommendation, or defer that part until the source can be reviewed. A clearly labeled document-finding aid can therefore receive lighter checking when it stays a finding aid. The boundary changes when its contents are used to justify analysis or action.

Sources: AI RMF Core | NIST; Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile | NIST

Can an AI detector tell you whether the summary is trustworthy?

No. NIST’s 2024 NIST GenAI (Pilot Study): Text-to-Text Evaluation Overview and Results tested whether systems could distinguish AI-generated summaries from human-written summaries. It did not test whether either kind of summary was factually correct. A detector result can therefore answer a question about likely authorship under a particular evaluation setup; it cannot verify the evidence in a policy summary or establish that its claims are trustworthy. The distinction is built into the study’s task. Published in June 2025 as NIST AI 700-1, the pilot used a curated dataset of article groups and associated human- and machine-generated summaries. Generator teams received topics and source documents, produced summaries, and discriminator teams then classified summaries as machine or human written. The report says the evaluation did not address differentiating “factual” from “non-factual” semantic content. In the discriminator task, systems received summaries without the source articles and were asked to score whether a large language model had generated the text. That setup tests origin classification, not whether a sentence follows from a cited source. The NIST publication record reports that detector performance varied significantly by systems: some generators could deceive most discriminators, while some discriminators could detect AI-generated content from almost all tested generators. The report also says discriminator systems improved over multiple rounds. Those findings matter for interpreting the benchmark, but they do not change its target. A detector may perform well on its defined classification task without testing whether a summary misstates a document, leaves out a qualification, cites a source that does not support the claim, or omits relevant evidence. Conversely, identifying a text as machine generated does not show that its claims are false. The pilot’s findings are bounded to its curated data, participating systems, and evaluation rounds; they are not a general verdict on every detector or every summary in current workplace use. This is a category error worth avoiding when time is tight: “Who or what likely wrote this?” and “What evidence supports this claim?” are separate questions. A policy analyst can resolve the second only by tracing a consequential statement to its cited material, checking that the material supports the wording, and considering its scope and qualifications. An authorship label cannot establish that a citation exists, that the source is relevant, or that the summary represents the evidence base. It also cannot replace the analyst’s judgment about whether a claim matters to the decision at hand. So do not use writing style or machine-origin classification as a proxy for accuracy. If a detector is part of a workplace process, treat its result only as information about the narrow classification task it was designed and evaluated to perform. For trust in a research summary, spend review effort on traceable sources, the fit between source and claim, and material limits. That conclusion is narrower than saying detectors never work: NIST found meaningful variation and performance on its benchmark. It is also more useful for this decision, because the benchmark did not test factuality.

Sources: 2024 NIST GenAI (Pilot Study): Text-to-Text Evaluation Overview and Results | NIST; 2024 NIST GenAI (Pilot Study): Text-to-Text Evaluation Overview and Results

What is one defensible next check before you pass the summary on?

Choose the sentence in the summary most likely to change the recommendation. Open its cited source and locate the passage, table, or result meant to support it. Compare the sentence with that evidence: does it describe the same population, period, and finding, and preserve qualifications that could alter interpretation? Record the outcome beside the claim: retained, narrowed to match the evidence, or unresolved. If the source is inaccessible or does not support the sentence, do not pass the claim on as established. Remove it, qualify it, or make the gap explicit while seeking a better source.

For the next reviewer, note the source location and one material limitation in the work artifact. That small record makes the reasoning inspectable without suggesting that one citation check proves the summary complete or correct. NIST’s Generative Artificial Intelligence Profile supports verifying sources and citations in system outputs and documenting fact-checking, especially for information from multiple or unknown sources. The Profile addresses organizational risk management; it does not prescribe this claim-by-claim sequence for policy analysts. The sequence here is an application of that guidance. NIST’s AI Risk Management Framework Core makes intended purpose, context, and human oversight relevant to organizational risk management, but sets no universal sign-off threshold for this summary. If a consequential source gap remains, carry it forward as uncertainty rather than letting fluent wording stand in for evidence.

Sources: Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile | NIST; AI RMF Core | NIST

Questions readers ask