Practical guide
Questions to ask when verifying AI-assisted work
Verification starts with the decision the work must support, not a generic instruction to “check the AI.” Ask: what must be true, which sources are authoritative, what transformations occurred, where could a material error hide, how will each failure be tested, who can resolve uncertainty, and what evidence proves the final output changed? The working artifact is a verification question ledger.
Verification starts with the decision the work must support, not a generic instruction to “check the AI.” Ask: what must be true, which sources are authoritative, what transformations occurred, where could a material error hide, how will each failure be tested, who can resolve uncertainty, and what evidence proves the final output changed? The working artifact is a verification question ledger connecting every material claim to a source, test, disposition, and release decision.
What must be true for the output to be usable?
Define the objective, audience, material claims, calculations, constraints, and consequence of error. A marketing brainstorm may tolerate speculative ideas if labelled. A financial narrative, policy interpretation, customer notice, or control recommendation requires stronger source authority and review.
Turn broad quality words into testable conditions. “Accurate” might mean figures reconcile to an approved extract and claims match cited passages. “Complete” might mean every mandatory requirement has a disposition. “Current” might mean the source version and access date are recorded.
The NIST Generative AI Profile identifies generative-AI risks and risk-management actions across the lifecycle. It is a voluntary profile rather than a checklist for every workplace output, but it supports explicit context, testing, documentation, and monitoring.
Sources: [1]
How will each material failure be tested?
Map claims to failure-specific checks. Verify citations by opening the source and reading the supporting passage. Reperform calculations independently. Compare extracted fields with a sample of originals. Test summaries against omitted and contradictory evidence. Check recommendations against stated constraints and the needs of the intended audience.
Record pass, fail, unresolved, and not tested. A page full of checkmarks can hide the most important untested condition. Define a stop rule for missing authority, unresolved material discrepancy, prohibited data, or a failed control.
Worked artifact: a synthetic supplier recommendation
Imagine a synthetic operations memo recommending a supplier based on price, delivery, and service history. The AI draft states that Supplier B is cheapest and has no late deliveries. The verification ledger links price to a spreadsheet and delivery history to a service report.
The reviewer finds that B's quote excludes freight and the service report covers only six months. Recalculated total cost makes Supplier A cheaper, while a second report shows two late deliveries by B in the prior period. The reviewer rejects both claims, updates the comparison, and changes the recommendation to a conditional selection pending a capacity check.
The ledger retains the original claims, source passages, calculations, failed tests, revised conclusion, unresolved capacity question, and release authority. The figures and suppliers are synthetic. The artifact does not prove procurement compliance or a business outcome.
Who resolves uncertainty and releases the work?
Name the source owner, subject-matter reviewer, risk or legal adviser where relevant, and final release authority. State what each person decides. Reviewers need enough time, source access, and authority to disagree. Escalation should preserve the uncertain claim rather than silently converting it into a confident answer.
Before release, ask whether the final artifact reflects every material disposition and whether someone else can reconstruct the reasoning. Reopen verification after a source, model, prompt, calculation, policy, or decision context changes.
These questions make verification specific and auditable. They do not establish universal correctness, certify controls, or remove the need for accountable judgment. Two Prune can provide task-bounded evidence of how participants use sources, challenge AI output, and revise work in simulations. It does not make employment decisions.
Sources
- 1.NIST Generative AI Profile · National Institute of Standards and Technology
- 2.GAO AI Accountability Framework · U.S. Government Accountability Office