Practical guide
How to report AI capability evidence to a learning and development leader
An L&D leader needs a report that supports a named programme decision, not a leaderboard of people or a collection of impressive examples. The report should connect the learning objective to observable work, show who and what the evidence represents, separate programme operation from assessment performance, and make missing or contradictory evidence visible. Use BRIEF to prepare it: bound the.
An L&D leader needs a report that supports a named programme decision, not a leaderboard of people or a collection of impressive examples. The report should connect the learning objective to observable work, show who and what the evidence represents, separate programme operation from assessment performance, and make missing or contradictory evidence visible. Use BRIEF to prepare it: bound the decision, report participation and process, interpret task evidence, expose limitations, and frame the next action. The output is a decision brief with an evidence annex, not a claim that training caused workplace outcomes.
R: Report reach and process before capability evidence
Show how the programme operated. Include eligible, invited, started, completed, withdrawn, technically blocked, and not observed counts. Keep denominators attached to every rate. Break down representation only where the categories are appropriate, lawful, sufficiently populated, and useful for the decision.
Report whether participants received the intended materials, tool access, time, accessibility support, instructions, and review process. Separate a delivery failure from an assessment result. If six people could not open a source document, their missing evidence cannot be converted into low capability.
Use a compact flow table:
| Stage | Count | Denominator | Material exception | |---|---:|---:|---| | Eligible | 42 | 42 | Two locations excluded by tool policy | | Invited | 36 | 42 | Six held pending approved access | | Started | 31 | 36 | Three scheduling conflicts, two declined | | Complete and reviewable | 27 | 31 | Four viewer-event gaps require separate handling |
Participation, completion, satisfaction, and confidence are useful programme measures. They are not substitutes for observed performance on a realistic task.
I: Interpret task evidence without collapsing constructs
Organise evidence around the work the programme intended to support. Describe the task, evidence opportunities, expected deliverable, and rubric. Report observed patterns across context framing, evidence navigation, AI orchestration, verification and risk control, judgment and synthesis, and communication and delivery when those dimensions were actually observable.
Keep direct observation separate from interpretation. A source was opened is an event. The participant noticed a contradiction is a reviewed interpretation supported by actions and the deliverable. The participant will verify future workplace outputs is not established by one simulation.
Avoid one composite number when it hides decision-relevant variation. Show distributions or bounded pattern counts, reviewer agreement where available, and examples selected by a stated rule. Do not feature only the strongest or most surprising cases. If early comparison groups are small or unstable, describe them as directional or unavailable rather than mature benchmarks.
E: Expose evidence strength, missingness, and alternative explanations
Attach a confidence note to every material conclusion. Explain the source evidence, missing data, reviewer process, sample limits, and plausible alternative explanations. A stronger final deliverable may reflect prior domain familiarity, better access to source materials, more time, or stronger general writing, not only the learning intervention.
The UK Government's guidance on impact evaluation of AI interventions recommends planning evaluation early and making the theory of change and context explicit. The Flexible AI Upskilling Fund evaluation distinguishes process and baseline evidence from later impact. Follow the same discipline: do not infer causation from a post-programme snapshot unless the design supports it.
List failures and contradictions in the main brief, not only the annex. Examples include incomplete event capture, inconsistent task versions, reviewer disagreement, inaccessible materials, unusual tool outages, or a sample that omits the people most affected by the proposed decision.
F: Frame the next action with conditions
End with a decision table. For each option, state what the current evidence supports, what it does not support, prerequisites, owner, date, and stop condition. Useful options may be stop, repair and rerun, continue within the same boundary, revise one learning component, or seek separate approval for an expanded pilot.
Do not recommend expansion merely because average scores rose or participants liked the experience. Connect the recommendation to the original decision and acceptance criteria. Keep later questions, such as workplace transfer, productivity, service quality, or return on investment, as separate evidence programmes.
Decision-brief artifact
| Section | Required content | |---|---| | Decision | Choice, authority, decision date, and alternatives | | Programme boundary | Population, objective, task, conditions, support, and exclusions | | Delivery flow | Eligible, invited, started, complete, reviewable, and exceptions | | Evidence | Construct, source, denominator, reviewer, observed pattern, and confidence | | Missingness | Missing source, cause, affected conclusion, and treatment | | Examples | Selection rule, task evidence, interpretation, and limitation | | Options | Stop, repair, continue, revise, or propose expansion | | Next gate | Owner, due date, prerequisite, and stop condition |
Worked example: reporting a verification-practice cohort
A synthetic organisation runs realistic practice for 36 policy analysts after introductory AI training. The programme decision is whether to add a coached verification module before the next cohort. L&D owns module design, the policy director owns work-quality criteria, and the programme sponsor owns any expansion.
Thirty-one participants start and 27 produce reviewable submissions. Across those 27, 19 identify a deliberately outdated source, 14 reconcile a denominator error, and 11 both correct the error and explain its decision consequence. Reviewer agreement is strong on source identification but weaker on whether uncertainty was communicated clearly. Four additional submissions are excluded from pattern counts because document-viewer events are missing; their final deliverables remain visible but cannot support process claims.
The brief does not say 70% of employees can verify AI outputs. It says that 19 of 27 reviewable submissions identified the planted outdated source under this task's conditions. It recommends adding a coached exercise on calculation consequences, repairing the viewer-event defect, clarifying one rubric anchor, and rerunning the same bounded task before considering a broader programme change.
Two Prune boundary
Two Prune can provide a realistic assessment simulation, capture available workflow and deliverable evidence, and report observed task performance with evidence-strength and calibration notes. It does not prove learning transfer, training effectiveness, productivity, return on investment, broad role readiness, or future workplace performance. It does not make employment decisions.
Primary sources
UK Government, Skills for AI: What works for AI upskilling in the UK
- UK Government, Guidance on the impact evaluation of AI interventions
- UK Government, Evaluation of the Flexible AI Upskilling Fund
Sources: [1] [2] [3]
Sources
- 1.UK Government, Skills for AI: What works for AI upskilling in the UK · UK Government
- 2.UK Government, Guidance on the impact evaluation of AI interventions · UK Government
- 3.UK Government, Evaluation of the Flexible AI Upskilling Fund · UK Government