AI Audit Response Checklist

Build the request-to-evidence register

Start with one row per auditor request, not one row per file. The U.S. Government Accountability Office AI accountability framework organizes oversight around governance, data, performance, and monitoring, and pairs practices with questions and audit procedures. Use those four domains as a coverage prompt, then preserve the auditor's exact request wording in the register.

For each row record:

The critical distinction is between a control description and operating evidence. A policy can show intended governance. It does not show that a review happened, that an exception was resolved, or that a control operated throughout the requested period. Likewise, a screenshot may demonstrate one state at one moment but not continuing operation. Label each item by what it actually supports.

• request ID and exact question;
• response owner and due date;
• the claim the response will make;
• system, version, environment, and time period in scope;
• evidence file, system-of-record link, and extraction date;
• evidence custodian and reviewer;
• exceptions, missing periods, and follow-up action;
• final disposition: ready, partial, not available, or not applicable.

Sources: [1]

Run the seven-check response test

Apply these checks to every register row before packaging it:

The NIST AI Risk Management Framework 1.0 is voluntary and use-case agnostic. Its Govern, Map, Measure, and Manage functions are useful as a completeness lens, not as a certification checklist. The NIST AI RMF Playbook offers suggested actions and documentation practices that organizations can tailor. Record which framework prompt informed a response, but do not write “NIST compliant” merely because a team used the framework.

• Scope match: Does the artifact cover the named model, workflow, business unit, environment, and period?
• Claim match: Can a reviewer point to the exact passage, field, or event that supports the response?
• Provenance: Is the source system, creator, extraction method, date, and version recorded?
• Completeness: Are missing samples, unavailable logs, excluded populations, and known exceptions explicit?
• Reproducibility: Could another authorized reviewer retrieve or regenerate the evidence using the recorded steps?
• Approval: Has the accountable control owner confirmed the statement and its limits?
• Disclosure: Does the response distinguish verified facts, management representations, estimates, and planned remediation?

Sources: [2] [3]

Worked response package

Suppose an auditor asks: “Show that human reviewers examine high-impact AI outputs before customer delivery during July–September.” The response owner creates register item AR-17 with a claim limited to one production workflow and that quarter.

The package contains: the approved review procedure version effective on 1 July; a system export listing all 184 in-scope outputs and their review states; a reproducible random-sample rule; 15 sampled records with timestamps and reviewer identifiers; an exception log showing four late reviews; and the remediation ticket for the scheduling defect. A cover note states that 180 records met the stated timing rule and four did not. It does not replace the exception with an average, call the process fully effective, or infer results for another workflow.

The reviewer traces the population count to the export, reruns the sample selection, checks that each sampled timestamp follows generation and precedes delivery, and confirms that the procedure was effective for the whole period. AR-17 is marked “partial” until the owner explains the four exceptions and the auditor decides whether more testing is needed.

Package and handoff rules

Create a read-only index with stable filenames such as AR-17-01-procedure.pdf and AR-17-02-population.csv. Keep originals in the governed repository; share controlled copies, not ad hoc edited duplicates. Redact secrets and personal data only under an agreed method, and record that redaction occurred. Hashes can help detect file changes, but a matching hash does not establish that the underlying control was appropriate or effective.

Before handoff, a person who did not assemble the row should perform the seven-check test. Return ambiguous requests to the auditor rather than guessing. Keep unresolved items visible, including evidence that cannot be produced. A concise, bounded “not available” is more auditable than a confident narrative unsupported by records.

Two Prune can be used to design realistic assessment exercises in which a participant assembles and explains an evidence package. The resulting assessment evidence describes observed performance in that simulation. It does not conduct an organizational audit, certify controls, or make an employment decision.