Verifying fabricated citations in AI-enabled knowledge work

Decompose every citation into testable parts

Extract the author or organization, title, date, publisher or journal, identifier, URL, and the exact claim the citation is meant to support. Preserve the wording from the draft before attempting repairs. Classify the source as primary research, official guidance, standard, policy, dataset, commentary, or another type relevant to the claim.

Test four questions in order: does the work exist; does the identifier resolve to that work; does the retrieved version contain the relevant passage or data; and does that evidence support the draft's actual wording and scope? A correct DOI does not prove the claim, and a matching title does not prove the cited edition is current.

The NIST Generative AI Profile identifies confabulation and information-integrity risks and recommends source verification and provenance-related practices. The profile is voluntary, cross-sectoral guidance; it does not establish that a generated citation is false or that a verification method catches every error.

Sources: [1]

Build the reference claim ledger

Use one row per claim-source pair. Record the draft claim, citation as supplied, identifier, resolved metadata, authoritative landing page, accessible version, relevant passage or table, population, period, method, limitations, support disposition, corrected wording, reviewer, and date.

Use distinct dispositions: verified support, real source but wrong metadata, real source but insufficient support, superseded, unresolvable, or remove. If a source supports only a narrower statement, revise the claim and retain the change. If the source cannot be inspected, label that limitation rather than citing the AI output as confirmation.

The DOI Handbook explains DOI identifiers, metadata, and resolution services. DOI resolution helps establish the identity and current location of an object, but reviewers still need to compare the source content with the claim. Not every valid source has a DOI, and not every DOI points to freely accessible full text.

Sources: [2]

Worked artifact: a synthetic workforce brief

A synthetic workforce brief states that a 2025 study found a 37 percent productivity increase across all knowledge workers. The supplied DOI resolves, but it belongs to a different paper about a narrow laboratory task. The title of the alleged study cannot be found on the named publisher site.

The ledger marks the identifier as real but mismatched and the claimed source as unresolvable. The reviewer searches the official publisher and registration metadata, finds no support for the universal result, removes the percentage, and rewrites the section to describe the organization's own proposed pilot measures without asserting an external outcome.

A second citation resolves to genuine guidance, but the cited passage discusses adoption rather than work quality. It is retained only for the adoption statement and removed from the quality claim. The example is synthetic and does not accuse a real paper, author, or tool of fabrication.

Verify the finished reference set

Have a reviewer who did not draft the text sample high-consequence and quantitative claims. Resolve identifiers from a clean browser session, compare author, title, date, and publisher, open the relevant version, and inspect the passage in context. Check retractions, corrections, superseding guidance, scope, denominators, and whether secondary reporting changed the original meaning.

Re-run link and metadata checks before publication and during scheduled refreshes, but do not equate HTTP success with evidentiary support. Retain rejected references and corrected claims so later revisions do not reintroduce them.

Two Prune can observe whether a participant verifies sources and narrows unsupported claims in a bounded assessment. It does not certify scholarship, establish factual truth for every domain, provide legal conclusions, or make employment decisions.