How to Report AI Capability Evidence to an AI Academy Owner

Start with the owner’s next decisions

List the decisions due in the next reporting cycle. An academy owner may decide whether to revise a module, add practice, commission another assessment wave, allocate coaching, or propose expansion. They may only influence budget, data retention, workforce policy, or operational deployment. Name the accountable owner for each decision instead of inferring authority from the academy title.

Create one report card per decision with: decision and deadline, target population, observation window, evidence available, evidence missing, interpretation boundary, recommended action, alternative action, owner, and next evidence gate. This prevents a dashboard metric from acquiring a meaning it was never designed to carry.

The OECD’s 2025 brief Bridging the AI skills gap distinguishes general AI literacy from advanced skills and says training supply, accessibility, inclusivity, and targeting remain important policy questions. The 2026 OECD brief AI and skills: What we know so far emphasizes monitoring changing skill needs and shared responsibility among employers, workers, and governments. These publications support careful programme context; they do not establish that a particular academy caused workplace outcomes.

Sources: [1] [2]

Build a signal ladder instead of a blended score

Use five separate rows:

For every row record population, numerator, denominator, collection method, task opportunity, reviewer, missingness rule, comparison, and claim limit. Never treat non-use as failure when access was unavailable, or absence of a behavior as weakness when the exercise provided no opportunity to demonstrate it. Show distributions and evidence strength rather than only averages. Small cohorts, self-selection, scorer disagreement, accessibility barriers, and instrumentation gaps belong beside the result they qualify.

NIST’s AI RMF Core calls for documented roles, context, operator proficiency, measurement, monitoring, and evaluation limits. Use those outcomes to structure the reporting controls. They are not a curriculum accreditation or a claim that an assessment predicts every workplace setting.

• Reach: who was invited, enrolled, attended, and completed, with eligible-population denominators.
• Experience: relevance, accessibility, and reported barriers, clearly labeled as self-report.
• Learning: knowledge or skill demonstrated under specified conditions.
• Applied-work evidence: observable framing, evidence use, AI orchestration, verification, judgment, and delivery in a realistic task.
• Transfer and operating outcomes: later behavior or operational measures, with the additional design needed to address attribution and confounds.

Sources: [3]

Produce a decision-trace report

Open with a one-page decision register. Each row should show the decision, evidence disposition, action, owner, and due date. Use dispositions such as act, investigate, hold, or not answerable. Then provide a cohort and access table so readers can see who is represented. Follow with signal-specific findings, representative evidence excerpts or artifact references, uncertainty, and a change log.

For applied-work evidence, include the task, materials, allowed tools, time limit, scored opportunities, review criteria, scorer calibration, and what telemetry was unavailable. Report recurrent behavior patterns with counts and denominators. Include counterexamples that challenge the headline. Do not publish raw participant material or personal data merely to make the report feel concrete.

Finish with an action ledger. Link every proposed curriculum change to a finding, an owner, a completion date, and the evidence needed to decide whether the change helped. Preserve “no decision” when evidence quality is inadequate.

Worked quarterly academy report

Consider a synthetic academy that invited 96 operations staff to a verification module. Seventy-two enrolled, 64 completed, and 48 entered a later evidence-checking simulation. Six simulation records are incomplete because the workspace logging service was unavailable, leaving 42 reviewable attempts.

The decision register contains three rows. First, “revise the source-verification lesson” receives an act disposition: 26 of 42 reviewable attempts repeat an unsupported claim, and artifact review shows that 18 of those participants opened the conflicting source but did not reconcile it. Second, “expand the academy company-wide” receives hold: the cohort was volunteer-heavy, no comparison design exists, and the exercise does not establish transfer or operating impact. Third, “add manager practice” receives investigate: participant comments identify inconsistent manager expectations, but comments alone do not show what support changes performance.

The report links the first finding to redacted artifact references and a scorer agreement check. It recommends a revised contradiction exercise followed by a pre-specified second wave. It reports completion as 64 of 96 eligible staff, simulation participation as 48 of 64 completers, and reviewable evidence as 42 of 48 attempts. It does not silently use 42 as the denominator for every academy claim.

The action ledger assigns the learning designer to repair the lesson, a functional reviewer to test realism, and the academy owner to approve the second wave. Expansion remains a separate proposal requiring authority and evidence beyond this report.

Run the reporting QA gate

Before release, ask a reviewer to trace every headline to its denominator, source population, artifact, and limitation. Recalculate rates; inspect missing records; test whether a different denominator reverses the message; compare examples with rubric criteria; and confirm that actions match the academy owner’s actual remit. Fail the report if it converts attendance into capability, a simulation into workplace performance, a self-report into an observed behavior, or correlation into programme impact.

Record the report version, source extracts, query or calculation logic, rubric version, scorer notes, redaction method, review disposition, owner, and next review date. If participant-level follow-up is permitted, state the purpose, access boundary, and retention rule. Use reader-facing language and keep employment decisions outside the academy report.

Two Prune can supply evidence from a defined assessment session, including submitted work and captured process signals, for an academy’s bounded reporting question. It does not prove training transfer, productivity, return on investment, general capability, or suitability for employment. Accountable people must interpret the evidence alongside access, curriculum, workplace, and governance context.