An AI Verification Template That Follows the Failure

Verification record template

Create one record for each material output. Include: decision supported; artifact version; claim or calculation; source and date; transformation performed; principal failure; test procedure; expected result; observed result; reviewer; exception; and release disposition. For narrative claims, record what the source supports and what remains interpretation. For calculations, preserve the input table and independent formula.

GAO’s data reliability guide focuses on accuracy, completeness, and applicability for the intended use (GAO data reliability assessment guide). Those dimensions can structure source checks, but they do not mean every verification can be reduced to data quality. Logic, permissions, currentness, and audience interpretation may require different tests.

Sources: [1]

Choose a test from the failure mode

Use reconciliation for missing or duplicated records, recalculation for arithmetic, source read-across for narrative claims, boundary tests for dates and units, counterexamples for brittle rules, and independent challenge for recommendations. When a tool can take actions, add a permission and destination check. When the source can change, record a snapshot or version identifier.

NIST’s AI RMF treats measurement and management as lifecycle activities rather than one-time certification (NIST AI RMF Core). That supports re-running tests when the system or context changes. The framework is voluntary and does not certify a completed template.

Sources: [2]

Worked template: quarterly performance note

Imagine a synthetic quarterly note drafted from a spreadsheet. The headline says conversion improved by 14 percent. The verification record identifies a percentage-point versus percentage-change risk. The reviewer traces the two period values, recalculates both forms, and determines that the draft used the wrong expression. A second check finds that one region was excluded from the current quarter but included in the prior quarter.

The corrected note states the comparable-population result, explains the exclusion, and links to the controlled table. The record stores the rejected headline, formulas, source versions, reviewer, and release decision. It does not ask the AI assistant to verify its own output using the same context and treat agreement as independence.

Exception and release fields

If a test fails, choose repair, qualify, remove, or hold. Record the owner and due date for any accepted exception. A release can be approved for a narrow audience while remaining prohibited for a broader one. Close the record only when the live artifact matches the reviewed version and the required evidence is retained.

Keep independence proportionate

Not every low-impact draft needs a second specialist, but a material claim should not be “verified” only by repeating the same prompt in the same model. Independence can come from a source-system query, a separate calculation, a reviewer with different incentives, or a controlled reference answer. State what kind of independence the check provides.

Version the completed template with the artifact. If the source, model, formula, prompt, or intended audience changes, identify which fields and tests must be reopened. A copied green status from an earlier version is not evidence about the new one.

This template is educational guidance. It cannot ensure accuracy, safety, control effectiveness, or compliance. Organizations must adapt tests to their actual systems, risks, policies, professional standards, and approval authorities. Missing evidence should remain visible rather than being scored as successful verification.

Audit a sample of completed records for evidence that the named test actually occurred. Repeated identical notes, absent attachments, or approvals before test timestamps indicate that the template may be functioning as paperwork rather than control evidence. Repair the workflow instead of adding more unchecked fields.