Practical guide

AI implementation plan verification checklist

Use this checklist to decide whether an AI implementation plan is ready for approval, needs targeted repair, or should be held. It tests whether the plan connects a bounded use case to accountable decisions and reproducible evidence. It does not certify an AI system, replace a security or legal review, or prove that a planned benefit will occur.

By Two Prune

Use this checklist to decide whether an AI implementation plan is ready for approval, needs targeted repair, or should be held. It tests whether the plan connects a bounded use case to accountable decisions and reproducible evidence. It does not certify an AI system, replace a security or legal review, or prove that a planned benefit will occur.

Prepare the review packet

Collect the plan, source manifest, workflow and data-flow diagram, risk register, control-to-test matrix, decision log, change log, and proposed monitoring record. Freeze the versions being reviewed. If a reviewer cannot identify which model, configuration, source collection, workflow, and plan revision the decision covers, mark the packet incomplete.

The NIST AI Risk Management Framework organizes work around Govern, Map, Measure, and Manage. NIST says these functions are not a universal ordered checklist and that risk management should continue throughout the lifecycle. Use them as a completeness test, then adapt the depth of review to the use case and its consequences. NIST AI RMF Core

Sources: [1]

The PLAN-TRACE checklist

Complete every row with pass, repair, hold, or not applicable, plus an evidence link and reviewer note. “Not applicable” needs a rationale and owner.

### P: Purpose and present-state evidence

Verification: ask a reviewer to explain what decision the evidence will support and what result would count against proceeding. Hold if the answer changes between reviewers.

### L: Limits and lineage

Verification: choose one output claim and trace it back through transformations to its source. Then remove or corrupt a source and confirm that the workflow fails visibly rather than silently substituting an unsupported answer.

### A: Accountability and authority

Verification: run a tabletop failure and ask who can stop the workflow immediately. A contact list without actual authority does not pass.

### N: Necessary controls and tests

The NIST AI RMF Playbook provides voluntary actions aligned to the four framework functions and explicitly says it is neither a checklist nor a sequence to follow in full. Select actions that fit the context and retain the rationale for exclusions. NIST AI RMF Playbook

### T: Transition and operational readiness

Verification: reconstruct a high-consequence case from input through final action, then demonstrate pause and recovery using the planned route.

### R: Review signals and response rules

### A: Acceptance and residual uncertainty

### C: Change and expansion gates

### E: Evidence integrity

  • [ ] The plan states one testable work outcome rather than “adopt AI” or “launch a tool.”
  • [ ] The current workflow, baseline, review burden, known defects, and unresolved evidence are recorded.
  • [ ] The proposed AI role is separated from the actions and decisions that remain human-owned.
  • [ ] Affected people, teams, geographies, systems, and external parties are named.
  • [ ] Expected benefits are hypotheses with a measurement method, not promised outcomes.
  • [ ] Permitted and prohibited uses are explicit.
  • [ ] Approved data classes, sources, vendors, integrations, and operating locations are listed.
  • [ ] Each decision-relevant input has an owner, origin, version, and update rule.
  • [ ] Sensitive, personal, confidential, licensed, or regulated material has a named handling route.
  • [ ] The plan says what happens when a source is stale, unavailable, contradictory, or outside scope.
  • [ ] The sponsor, workflow owner, evidence owner, risk reviewers, support owner, and decision authority are named.
  • [ ] Approval, exception, pause, rollback, and retirement authority are assigned.
  • [ ] Training and competency requirements are linked to the people performing each control.
  • [ ] An affected user has a route to report a problem or request human review where relevant.
  • [ ] Conflicts of interest and independence limits in testing are recorded.
  • [ ] Every material requirement maps to a control, test, evidence artifact, acceptance rule, and owner.
  • [ ] Material defect classes are evaluated separately rather than hidden in one average score.
  • [ ] Representative, boundary, adversarial, accessibility, recovery, and misuse cases are included where applicable.
  • [ ] Missing coverage is reported as missing evidence, not a passing result.
  • [ ] Retesting rules cover model, prompt, source, data, integration, user-population, and policy changes.
  • [ ] Pilot scope, user eligibility, support coverage, and escalation routes are defined.
  • [ ] Access control, logging, rollback, incident response, records handling, and recovery have been rehearsed.
  • [ ] The production or pilot boundary prevents unapproved sending, publishing, or consequential action.
  • [ ] Dependencies and service failures have fallbacks with named owners.
  • [ ] The launch sequence includes a reversible first step and evidence review before expansion.
  • [ ] Monitoring covers material defects, overrides, complaints, exceptions, source failures, drift, latency, and unauthorized use as relevant.
  • [ ] Each signal has a review frequency, threshold or judgment rule, and disposition owner.
  • [ ] The plan distinguishes observation from control by stating what action follows a signal.
  • [ ] Exceptions have a rationale, compensating control, affected scope, owner, and expiry.
  • [ ] Periodic review occurs even when no incident is reported.
  • [ ] Acceptance rules were frozen before results were reviewed.
  • [ ] Critical failures cannot be averaged away by stronger performance elsewhere.
  • [ ] Residual risks, untested contexts, evidence gaps, and transfer limits are visible to the decision authority.
  • [ ] The disposition is recorded as proceed, restricted pilot, repair and retest, pause, or stop.
  • [ ] The next decision date and evidence required for it are recorded.
  • [ ] Expansion to a new team, data class, geography, source, model, or action requires explicit evidence.
  • [ ] Configuration changes are versioned and linked to targeted retesting.
  • [ ] The plan identifies cumulative risk from several individually small changes.
  • [ ] Retirement includes access removal, record handling, dependency cleanup, and communication.
  • [ ] Lessons and rejected assumptions are retained for the next review.
  • [ ] Test inputs, outputs, evaluator decisions, defects, retests, deviations, and environment details are reproducible.
  • [ ] Evidence access is controlled and retention is proportionate.
  • [ ] Claims in the approval summary point to the underlying evidence rather than a slide-only conclusion.
  • [ ] A reviewer independent of the author has challenged the weakest assumption.
  • [ ] The final record distinguishes source fact, inference, decision, and unresolved question.

Sources: [2]

Copyable review record

| Area | Disposition | Evidence | Defect or gap | Owner | Due date | Retest trigger | | --- | --- | --- | --- | --- | --- | --- | | purpose and baseline | pass / repair / hold / N/A | link | note | name | date | change | | limits and lineage | pass / repair / hold / N/A | link | note | name | date | change | | accountability | pass / repair / hold / N/A | link | note | name | date | change | | controls and tests | pass / repair / hold / N/A | link | note | name | date | change | | operational readiness | pass / repair / hold / N/A | link | note | name | date | change | | monitoring and response | pass / repair / hold / N/A | link | note | name | date | change | | acceptance and change | pass / repair / hold / N/A | link | note | name | date | change |

Worked example: support-summary pilot

A team proposes a drafting assistant for customer-support summaries. The first packet has a timeline, vendor name, and “90% acceptable” target. PLAN-TRACE finds that account data is mixed with approved test cases, no one can pause the integration, and unsupported policy claims are included in the aggregate score.

The reviewer marks limits and lineage hold, accountability repair, and necessary controls repair. The revised plan narrows the pilot to synthetic and approved non-sensitive cases for one product. The assistant cannot send. A support quality lead receives stop authority. Unsupported policy statements and missed escalations become separate critical defect classes. The team adds source links, a frozen case set, rollback rehearsal, and a 30-day exception expiry.

The final disposition is restricted pilot, not general launch. Expansion requires a second evidence review, acceptable critical-defect results, manageable review time, and no unresolved high-severity incident. This worked result shows why checklist completion is not the goal: the checklist changes a decision by making missing authority and evidence visible.

Two Prune can use implementation-plan review as a realistic assessment task and describe how a participant frames the work, navigates evidence, uses AI, verifies claims, exercises judgment, and communicates a disposition. It does not certify the implementation, promise outcomes, or make employment decisions.

Sources

NIST AI Risk Management Framework Core

  • NIST AI Risk Management Framework Playbook

Sources: [1] [2]

Sources

  1. 1.NIST AI RMF Core · National Institute of Standards and Technology
  2. 2.NIST AI RMF Playbook · National Institute of Standards and Technology