AI Capability Measurement Guide for the AI Governance Leader

Define the governance leader's role in each decision

Start by classifying the leader as owner, reviewer, influencer, or collaborator for each decision. Governance titles do not establish budget authority, operational ownership, legal accountability, or control operation. Record the actual mandate, decision rights, escalation route, and required co-owners.

NIST's AI Risk Management Framework assigns executive leadership responsibility for decisions about AI risks and calls for clear roles, communication lines, policies, monitoring, and periodic review. It also treats governance as a cross-cutting function rather than a one-time approval. Use those outcomes to map accountability, not to assume that one governance leader performs every task. NIST AI RMF Core

Create a mandate card with five fields: decision, role, authority, required partners, and evidence needed. If authority is unclear, resolving that ambiguity is the first agenda item.

Sources: [1]

Build a seven-decision agenda

Organize the agenda around recurring decisions rather than available dashboards:

For every decision, name the deadline, accountable owner, affected population or system, consequence of delay, minimum evidence, contradictory evidence, and next collection step. Permit “unknown” as a valid status. A governance agenda that cannot represent missingness encourages false precision.

• Which AI-enabled work is in scope and why?
• Which uses require prohibition, additional control, or bounded experimentation?
• What human responsibilities must remain visible?
• What evidence is needed before expansion, repair, or pause?
• Which incidents, exceptions, or weak signals require escalation?
• How will training, assessment, and operating evidence be reviewed together?
• When should policies, controls, and the evidence plan be revised?

Separate the evidence layers

Keep six layers distinct: inventory and access, training participation, tool activity, task-bounded capability evidence, control-operation evidence, and operating outcomes or incidents. Each layer answers a different question. Attendance does not show applied judgment; assessment performance does not show that a production control operates; an incident count does not reveal exposure or reporting quality.

For each layer, record source, denominator, time window, collection method, owner, missingness, and known confounds. Use stable evidence identifiers so a governance decision can be reconstructed later. The NIST Govern Playbook describes policies and procedures as context-dependent mechanisms for systematizing roles and responsibilities across the lifecycle. It is a source for designing the governance process, not validation of a specific organization's evidence. NIST AI RMF Govern Playbook

Sources: [2]

Create the decision-evidence map

Use one row per decision. Columns should include required evidence, available evidence, evidence against the proposed action, confidence, affected groups, control dependencies, owner, approval route, and next review. Link every material statement to a source or mark it as a hypothesis.

Add a transfer column for capability evidence. It states how closely the observed task matches the work context and what important conditions differ. A realistic simulation may reveal verification behaviour under a known task; it does not by itself establish performance in every workplace setting.

AI can organize source registers, compare versions, locate empty cells, and draft alternative interpretations. Require it to return citations and uncertainty. The governance leader and relevant domain, security, legal, privacy, and operational owners decide what the evidence means.

Assign actors without collapsing accountability

Map the work across governance and oversight, domain expertise, impact assessment, operations, procurement, and independent evaluation where relevant. NIST's AI actor descriptions distinguish governance and oversight from domain, evaluation, procurement, and other lifecycle tasks. This supports a multi-actor map and cautions against treating governance as a single-person function. NIST AI actor task descriptions

Use a challenge rule for material decisions: the proposer cannot be the only reviewer, and the reviewer must have access to the evidence and authority to record disagreement. Document how unresolved disagreements are escalated. Independence should be defined for the context rather than implied by a job title.

Sources: [3]

Worked agenda: expanding an internal assistant

Suppose an organization is considering expansion of an internal AI assistant from 80 service staff to 420. The governance leader is the decision reviewer, while the service owner owns expansion and security owns access controls. Available evidence shows 61 trained users, 47 active users, 30 completed simulated service tasks, six double-reviewed submissions, two prompt-handling exceptions, and no stable definition of a production quality incident.

The agenda does not combine those figures. It records strong access and activity evidence, limited task-bounded capability evidence, a weak independent-review base, observed handling exceptions, and an outcome layer that is not measurable yet. The decision-evidence map recommends repairing the incident definition, reviewing the two exceptions, sampling more submissions independently, and running a constrained expansion with an explicit stop rule.

The report does not claim the assistant is safe, that staff are ready for every task, or that the assessment predicts operating results. It shows why the current evidence supports a bounded next step rather than full expansion or indefinite delay.

Set the cadence and decision record

Tie cadence to decision frequency and risk, not reporting habit. Maintain a monthly evidence review for changing exposures, an event-triggered review for incidents and exceptions, and a periodic policy review with named owners. Each meeting should end with a decision, hold, evidence request, or explicit no-change disposition.

The decision record should preserve the question, authority, sources, contradictory evidence, confidence, dissent, action, owner, due date, and next review. Track whether requested evidence was collected and whether the later decision changed. This makes governance learning visible without pretending that activity volume equals effectiveness.

Two Prune can contribute structured, task-bounded assessment evidence about how a participant handled a realistic AI-enabled assignment. It does not operate an AI governance programme, certify controls or compliance, determine organizational readiness, prove transfer to production work, or make employment decisions. Human owners must interpret the evidence alongside system, policy, incident, and operating evidence.