Practical guide
How to create a compliance matrix with AI
A compliance matrix maps source text to scope, action, owner, and proof. AI may extract text, while people decide meaning, fit, review, approval, and status.
A compliance matrix maps source text to scope, action, owner, and proof. AI may extract text, while people decide meaning, fit, review, approval, and status.
Define the decision before opening the source
Write a one-sentence purpose such as: “Map the voluntary NIST AI RMF outcomes relevant to the customer-support assistant pilot so the programme owner can identify evidence gaps before launch.” Name the system or workflow, business unit, geography, lifecycle stage, source edition, assessment date, reviewer, and excluded questions.
The useful output is not a spreadsheet filled with green cells. It is a controlled record that lets a reviewer trace each material assertion back to a current source and a real artifact.
This scope prevents a common failure: importing every sentence from a framework and treating the result as a universal checklist. NIST describes its AI RMF Playbook as voluntary and adaptable. Its suggestions are not comprehensive or ordered implementation steps. A matrix using that source should preserve those qualifications rather than turning them into mandatory claims. NIST AI RMF Playbook
If the source is a law, contract, certification scheme, or regulator's rule, record the jurisdiction, effective version, covered entity or activity, and the person qualified to decide applicability. AI-assisted extraction is not legal review.
Sources: [1]
Use a CLAIM-EVIDENCE row structure
Create one row per independently reviewable source statement. Use these fields:
| Field | Required content | | --- | --- | | Citation | Stable source, section, edition, date, and exact passage or bounded summary | | Lens | Requirement, guidance, definition, evidence request, or interpretation | | Applicability | In scope, out of scope, conditional, or unresolved, with a reason and reviewer | | Action | The specific process, control, decision, or evidence needed | | Implementation | Current, partial, planned, absent, or not assessed | | Material | Link to the actual policy, test, approval, record, or operating artifact | | Evidence test | How a reviewer will confirm that the material addresses the row | | Owner | Person accountable for the response and next review date | | Notes | Assumptions, exceptions, dependencies, and unresolved questions |
Keep source text separate from organizational interpretation. Label direct requirements, voluntary guidance, internal decisions, and reviewer inferences differently. A source citation does not establish that a control exists, and a policy document does not establish that the control operates.
The U.S. Government Accountability Office's AI Accountability Framework organizes practices around governance, data, performance, and monitoring and supplies questions and audit procedures. That structure is useful for testing a matrix response because it connects assertions to reviewable evidence rather than self-attestation alone. It does not make a matrix an audit opinion. GAO AI Accountability Framework
Sources: [2]
Give AI bounded extraction work
Provide the approved source file, its edition, the matrix schema, the defined scope, and instructions to retain section identifiers. Ask AI to propose candidate rows, quote or accurately summarize the relevant passage, flag cross-references, and identify missing fields. Do not ask it whether the organization complies.
Review extraction in two directions:
Then check every applicability decision and evidence link manually. Duplicate rows should be consolidated only when their source meaning, owner, action, and evidence test are truly the same. Similar wording can hide different conditions.
- Source to matrix: sample source sections and confirm that every relevant statement reached a row.
- Matrix to source: sample rows and confirm that the citation resolves and supports the recorded meaning.
Worked artifact: support-assistant pilot
Consider a synthetic organization mapping selected NIST AI RMF outcomes for an internal support assistant. The team is not seeking certification. It wants a launch-readiness evidence map.
| Citation | Lens | Applicability | Action | Implementation | Material | Evidence test | Owner | | --- | --- | --- | --- | --- | --- | --- | --- | | GOVERN 2.1 | Guidance on documented roles and communication | In scope: pilot has four operating roles | Approve a role and escalation map | Partial | Draft RACI and on-call route | Choose three planted events and confirm each reaches an owner with authority | Programme owner | | MAP 1.1 | Guidance on intended purpose and context | In scope: allowed and disallowed uses determine testing | Record users, tasks, boundaries, impacts, assumptions, and prohibited uses | Current | Pilot scope v3 | Compare ten evaluation tasks and five user prompts with the approved scope | Product owner | | MEASURE 1.1 | Guidance on selecting measures for mapped risks | Conditional: only mapped risks with observable tests | Link each priority risk to a measure or record why it cannot be measured | Planned | Evaluation plan draft | Trace every high-priority risk to a method, threshold, owner, and result field | Evaluation lead | | MANAGE 1.3 | Guidance on planned responses to high-priority risks | In scope: pilot needs stop and remediation routes | Define response options and approval authority | Absent | None | Run a tabletop event and retain the decision record | Risk owner |
The first row must not be marked “complete” merely because a RACI exists. Its evidence test may reveal that the security contact can advise but cannot pause the pilot. The fourth row remains absent until a response procedure and authority are implemented and exercised. These are synthetic statuses, not evidence about Two Prune or a customer.
Apply acceptance criteria before release
Reject a matrix when a reviewer cannot reproduce its source mapping, applicability logic, or evidence test. At minimum, confirm that:
AI can accelerate candidate extraction, comparison, formatting, and missing-field checks. Human owners remain responsible for source selection, interpretation, applicability, evidence sufficiency, and release. A compliance matrix is an evidence-organizing artifact. It does not certify compliance, prove control effectiveness, or replace legal, regulatory, audit, or subject-matter review.
Two Prune may use a synthetic matrix task to assess how a participant navigates evidence, verifies mappings, handles uncertainty, and communicates gaps. It does not operate a compliance programme or make employment decisions.
- every source has an edition, date, owner, and refresh trigger;
- material passages are not reduced to misleading fragments;
- applicability decisions name their reason and qualified reviewer;
- implementation status is separate from evidence strength;
- each evidence link resolves to a controlled artifact, not a vague folder;
- gaps, exceptions, and unresolved interpretations remain visible;
- source and row samples have been checked in both directions;
- the final matrix records author, reviewer, approval state, and next review date.
Sources
NIST: AI Risk Management Framework Playbook
- NIST: AI Risk Management Framework Core
- U.S. GAO: Artificial Intelligence Accountability Framework
Sources: [1] [3] [2]
Sources
- 1.NIST AI RMF Playbook · National Institute of Standards and Technology
- 2.GAO AI Accountability Framework · U.S. Government Accountability Office
- 3.NIST: AI Risk Management Framework Core · National Institute of Standards and Technology