How to assess hallucination detection in AI-enabled work

Define the support boundary before checking the output

NIST calls confidently presented erroneous or false generative-AI content “confabulation” and notes that it can include invented citations, contradictory statements, and content that diverges from the prompt. The UK Government's generative-AI framework similarly warns that language models generate statistically likely language rather than reliable accounts of reality. These sources support a practical lesson: fluency is not evidence.

This is narrower than general fact checking because the focus is the boundary between what the model produced and what the task evidence can support. A strong worker does not merely ask whether an answer sounds plausible. They trace decision-relevant statements to permitted evidence, test suspicious details, and prevent unresolved content from reaching the final deliverable.

Start with a support boundary. It lists what the worker may rely on for this task:

Anything outside that boundary is not automatically false. It is unresolved until checked. This distinction matters because a detector that labels every unfamiliar statement as wrong will punish useful inference, while a detector that accepts plausible prose will miss fabricated names, dates, quotations, relationships, and sources.

• the supplied documents, data, messages, and approved systems of record;
• current authoritative external sources that the task permits;
• calculations that can be reconstructed from retained inputs;
• explicit assumptions, clearly marked as assumptions rather than facts; and
• original recommendations that are presented as judgment, not attributed evidence.

Use the TRACE method

Apply TRACE to each decision-relevant statement in a generated draft:

The output is a support map, not a confidence score. A worker should be able to show why each material statement is retained and what happened to unsupported content. Model self-critique may help generate suspects, but it cannot close the check because the same system can confidently defend its own invention.

• Tag the statement as supplied fact, externally sourced fact, calculation, assumption, quotation, or recommendation.
• Route it to the evidence object that should support it, such as a contract clause, dataset row, policy owner, or current primary source.
• Align the statement with that evidence. Check entity, date, quantity, scope, condition, and attribution rather than accepting a shared topic.
• Challenge details that have no route, conflict with another record, or appear more specific than the source.
• Expose the disposition in the work product: retain, correct, qualify, remove, or escalate.

Worked artifact: a support map for a supplier briefing

Consider an original synthetic exercise. A participant receives a supplier proposal, an internal security note, a pricing sheet, and a meeting email. An AI assistant drafts a briefing with these statements:

The participant creates this TRACE support map:

| Statement | Tag and route | Alignment test | Disposition | | --- | --- | --- | --- | | Supplier history since 2022 | Supplied fact; proposal company profile | No internal record or cited source confirms the relationship | Remove and ask the supplier for evidence | | ISO 27001 certification | Externally verifiable status; certificate register or current certificate | Proposal says “aligned with ISO 27001” but names no certificate | Correct to “the supplier describes its controls as aligned” and hold any certification claim | | £184,000 including migration | Calculation; pricing sheet | Base fee is £184,000, while migration is a separate £22,000 line | Correct total and show the calculation | | Security approval on 18 September | Current status; internal security note | Note records a review meeting and two open conditions, not approval | Replace with the review status and open conditions |

The participant then searches the summary, table, and recommendation for the same unsupported details. This catches propagation: removing a fabricated history sentence is not enough if “trusted incumbent” remains in the recommendation.

All names, dates, amounts, and records in this example are synthetic. They are not customer evidence, product outcomes, or benchmarks. The artifact demonstrates how hallucinations can mix invented details with misread real evidence.

• “Northstar Systems has supported the programme since 2022.”
• “The proposed service is certified to ISO 27001.”
• “The annual price is £184,000, including migration.”
• “Security approved the hosting arrangement on 18 September.”

Design an assessment that reveals the skill

Give the participant a bounded source pack and an AI-generated draft containing several different traps:

Ask for the revised deliverable plus the TRACE support map. Midway through the exercise, introduce a new source that resolves one held statement and contradicts another. This tests whether the participant can update the evidence state rather than treating detection as a one-time proofreading pass.

Observe the workflow as well as the final prose. Useful evidence includes whether the participant opened the relevant files, followed citations to their actual targets, reconstructed a calculation, searched for repeated unsupported details, and preserved an uncertainty marker. The number of items flagged is not a sufficient score. Blanket scepticism can look busy while failing to distinguish supported surprise from unsupported plausibility.

• an entirely invented entity or event;
• a real source with a fabricated quotation;
• an accurate number attached to the wrong period;
• a plausible causal explanation absent from the evidence;
• a recommendation that is legitimate judgment when labelled correctly; and
• a supplied fact that looks surprising but is genuinely supported.

Keep hallucination detection distinct from adjacent controls

Hallucination detection asks whether generated content crosses the support boundary. Citation verification asks whether a named source is authentic, current, and supportive. Fact checking tests a precise claim against the best available authority even when the claim came from a human. Contradiction detection finds incompatible statements within or across records. These controls can be combined, but their evidence artifacts differ.

For example, an invented citation is first a hallucination suspect. Citation verification then shows that the cited object does not exist. A claim copied accurately from a real but obsolete policy is not necessarily a hallucination; it is a source-authority or fact-checking failure. Preserving these distinctions makes the assessment diagnostic rather than merely punitive.

Score observable decisions and close the loop

Assess performance across five dimensions:

Before release, compare the final deliverable with the support map. No held item should reappear as fact, and no correction should remain isolated in one paragraph. Record the version of the source pack and the generated draft so the reviewer can reproduce the check.

Hallucination detection reduces the chance that unsupported generated content reaches a reader. It does not establish that every retained source is correct, that every relevant source was available, or that the finished work is safe for every use. NIST presents confabulation within a wider set of generative-AI risks and risk-management actions, not as a problem solved by one checklist.

In a Two Prune assessment, the TRACE support map, source navigation, dispositions, and propagated revisions can provide task-bounded evidence about Verification & Risk Control, Evidence Navigation, and Communication & Delivery. They do not certify professional competence, predict workplace outcomes, or make an employment decision.

• coverage: material generated statements were tagged rather than sampled casually;
• routing: each statement was connected to the right evidence object or marked unresolved;
• alignment: entity, time, quantity, scope, and attribution were tested;
• disposition: unsupported content was corrected, qualified, removed, or escalated;
• propagation: changes reached headings, summaries, tables, calculations, and recommendations.

Sources

NIST: Artificial Intelligence Risk Management Framework, Generative Artificial Intelligence Profile

• UK Government: Generative AI Framework for HMG
• Government of Canada: Guide on the use of generative artificial intelligence

Sources: [1] [2] [3]