Practical guide
AI capability measurement guide for the learning and development leader
The learning and development leader's useful question is not “Which AI course should we buy?” It is “What capability decision must we make, and what evidence would let us make it responsibly?” Course completion, confidence surveys, manager observations, and work simulations answer different questions. Use the DECIDE agenda to connect the L&D mandate to one choice, compare evidence options, and.
The learning and development leader's useful question is not “Which AI course should we buy?” It is “What capability decision must we make, and what evidence would let us make it responsibly?” Course completion, confidence surveys, manager observations, and work simulations answer different questions. Use the DECIDE agenda to connect the L&D mandate to one choice, compare evidence options, and define a bounded pilot. Local governance determines whether the L&D leader owns, influences, reviews, or collaborates on the decision; the title alone does not confer budget or approval authority.
D — Define the mandate and decision
Begin with one written decision, not a general ambition to “measure AI skills.” Examples include:
Record the decision owner, L&D's role, affected population, timing, constraints, and what is explicitly out of scope. Avoid combining programme selection, individual evaluation, workforce planning, and performance management in one evidence exercise.
The UK government's 2026 employer guide says AI upskilling should be aligned to organisational goals, tailored to roles and starting points, supported by leadership, and evaluated. Its accompanying methodology also makes clear that evidence quality varies. That supports a decision-led agenda, not a claim that one learning format works everywhere.
- which practice pathway to fund for a defined population;
- whether people can apply a taught workflow in a realistic task;
- where additional verification or risk-control practice is needed;
- whether evidence is usable enough to expand a learning intervention.
E — Establish the capability construct
Define the observable capability before choosing an instrument. The UK AI Foundation Skills benchmark distinguishes technical and non-technical foundation skills and describes practical expectations for workers. An organisation should translate relevant expectations into its own work context.
For an AI-enabled knowledge-work assessment, a usable construct might cover:
Create a coverage map showing which behaviour is observed by each measure. If an important behaviour has no observation, label the gap rather than filling it with a proxy.
- framing the task, audience, constraints, and success criteria;
- finding and weighing relevant evidence;
- directing and refining AI support;
- checking claims, calculations, uncertainty, and operational risk;
- synthesising a decision or recommendation;
- communicating a useful deliverable.
C — Compare evidence alternatives
Use the same decision criteria for every option.
| Evidence option | Useful for | Principal limitation | |---|---|---| | Course completion | Access and participation records | Does not show application in work | | Knowledge check | Recall of defined concepts | May miss judgment and tool use | | Confidence survey | Perceived readiness and barriers | Self-report can diverge from behaviour | | Manager observation | Contextual examples over time | Opportunity and rater consistency vary | | Work sample or simulation | Observable process and deliverable in a controlled task | Requires task validity, scoring quality, accessibility, and careful interpretation | | Operational outcome | Changes in real workflows | Attribution is difficult and may arrive late |
Ask whether each option is relevant to the decision, proportionate, accessible, repeatable, interpretable, and feasible. A mixed design often answers more than a single score: participation can show reach, a simulation can show observed application, and follow-up observation can test whether behaviours appear in normal work. None should be presented as evidence for a question it did not measure.
I — Identify access, governance, and objections
Resolve these questions before inviting participants:
Common objections should change the design. “This will become surveillance” calls for minimised collection, clear purpose, access limits, and participant communication. “One exercise cannot represent a role” calls for a narrower claim and, where material, multiple evidence occasions. “Managers already know” calls for a coverage comparison, not dismissal of contextual observation.
- Who can access raw activity evidence, scores, summaries, and aggregate reporting?
- What is the retention period, and who can correct an identity or data error?
- What accommodations, assistive technologies, language support, or alternative formats are needed?
- Are participants told the purpose, permitted tools, timing, data use, and limits of interpretation?
- What happens when technical failure or an inaccessible interaction affects the evidence?
- Which groups are too small for useful aggregate reporting?
D — Design a bounded pilot
Specify the pilot before launch:
Do not set expansion as the default. Predefine possible dispositions such as stop, repair and retest, continue unchanged, or expand with stated conditions.
- one decision and one defined population;
- a representative but bounded task;
- the measures, scoring rules, and evidence exclusions;
- accessibility and technical checks;
- owners for administration, scoring, review, privacy, and incident response;
- stop conditions for missing evidence, scoring ambiguity, access failure, or participant harm;
- a comparison with the current evidence method;
- a retest or repair path.
E — Evaluate and report for the decision
Reconcile invitations, starts, completions, exclusions, and analysed records. Report distributions and missingness where the group size supports them; do not rely on a single average. Separate observed evidence from interpretation and recommendations. Record scoring disagreements, technical incidents, accommodations, and evidence gaps.
The final brief should answer:
- What decision was tested?
- What evidence was collected, and from whom?
- What did each measure actually observe?
- What failed, remained uncertain, or could not be interpreted?
- Did the evidence change the decision?
- What is the next bounded action and its owner?
Worked agenda: a 60-day service-team decision
A synthetic L&D team must decide whether to fund guided AI practice for 80 customer-operations staff. The L&D leader owns programme design but is an influencer on assessment selection; operations owns workflow standards, and the data-protection lead reviews evidence handling.
The team uses DECIDE:
The disposition is repair and retest, not expansion. The team fixes the interaction, clarifies the verification evidence anchor, and schedules a second small run. The finding supports a programme-design decision for this context; it is not a claim of organisation-wide readiness, productivity impact, or transfer to every role.
- Define: choose between a generic course, role-based practice, or no new programme for this quarter.
- Establish: map the six behaviours above to a case involving an ambiguous service escalation, internal evidence, AI-supported drafting, and source checking.
- Compare: retain course completion for reach, add a structured simulation for application, and schedule a later manager observation for normal-work context. A confidence survey is used only to identify perceived barriers.
- Identify: restrict raw interaction evidence to authorised reviewers, publish the retention period, test keyboard navigation, and provide a documented incident route.
- Design: pilot with 16 volunteers across two shifts. Stop if more than two records lack required evidence, if a material accessibility defect appears, or if scorers cannot resolve a rubric ambiguity.
- Evaluate: 15 complete; one technical interruption is excluded. Reviewers find that verification behaviour is inconsistently evidenced, while completion data alone would have hidden that issue. A keyboard-focus defect also appears.
Boundaries for Two Prune evidence
Two Prune helps customers run realistic AI-enabled work simulations and produce evidence-backed assessment reports about observed performance. It does not establish the right curriculum by itself, prove learning transfer, certify AI readiness, determine compensation or employment outcomes, or replace human review. Early or small benchmark groups should be described as directional or unavailable, with score-confidence and calibration notes where evidence is limited.
Primary sources
UK Government, Employer guide: what works for AI upskilling in the UK
- UK Government, Research evidence, analysis and methodology: what works for AI upskilling in the UK
- UK Government, AI Foundation Skills for Work benchmark
Sources: [1] [2] [3]