Practical guide
AI capability pilot design for an HR leader
An HR-led capability pilot should be a small decision experiment, not a miniature rollout or a search for an impressive average score. It should create comparable opportunities to demonstrate a defined behaviour, expose access and operating problems, and end with a preset choice to stop, repair, repeat, or expand cautiously. The useful artifact is a pilot protocol that fixes the decision, cohort.
An HR-led capability pilot should be a small decision experiment, not a miniature rollout or a search for an impressive average score. It should create comparable opportunities to demonstrate a defined behaviour, expose access and operating problems, and end with a preset choice to stop, repair, repeat, or expand cautiously. The useful artifact is a pilot protocol that fixes the decision, cohort rationale, task, evidence, responsibilities, review method, reporting boundary, and exit gate before participation begins.
Design the pilot with TRIAL
Use Target the decision, Recruit for coverage, Instantiate the work, Assure operation, Lock the exit rule. This differs from an HR decision agenda, which frames questions, and from a business case, which appraises investment options. The pilot protocol controls one bounded test in operation.
The UK government's 2026 employer guide to AI upskilling describes practical, contextualised, accessible, integrated, and sustainable learning based on workshops, case studies, and a national employer survey. Its methodology report makes the evidence base and contextual variation visible. NIST's AI Risk Management Framework actor-task appendix separates operation, monitoring, test, evaluation, verification, and validation responsibilities across the lifecycle. These sources support realistic tasks, access checks, defined roles, and monitoring; they do not prove that one pilot design transfers to every workplace. UK employer guide to AI upskilling UK AI-upskilling research methodology NIST: Descriptions of AI actor tasks
### T: Target one decision
Write the decision the pilot must inform and the person accountable for it. Examples include whether to repair a learning module, repeat practice for a specific behaviour, test another task context, or stop because evidence is not useful enough. Avoid “measure AI capability” as the objective.
Define HR's role for this pilot as owner, influencer, reviewer, or collaborator. HR may own learning design, participant communication, accessibility, use boundaries, and the feedback route. The functional owner defines realistic work and quality. The assessment owner governs the rubric and evidence. Security and privacy owners govern data and access. The sponsor owns the continuation decision.
State prohibited uses. A development pilot should not become a hidden selection exercise, a league table, or an automatic employment decision. The protocol should say whether participant-level evidence can be seen, by whom, for what purpose, and for how long.
### R: Recruit for evidence coverage
Choose participants to cover the variation needed for the decision, not to produce a representative percentage from a convenient group. Record the source population, invitation route, eligibility, exclusions, consent or notice, accommodations, prior learning exposure, relevant experience, and non-participation.
Include enough variation to reveal task and operating problems: different starting points, access needs, work contexts, or prior tool exposure where relevant. Do not infer weakness from non-participation or missing evidence. Track invitation, start, completion, technical failure, withdrawal, and assessable submission separately.
If the pilot uses volunteers, report that limitation. If managers nominate participants, examine whether nomination changes the evidence opportunity. Avoid percentage claims when the denominator is unclear or the cohort is too small for the intended inference.
### I: Instantiate a realistic bounded task
Name the task, audience, inputs, constraints, time, AI support, expected deliverable, and principal failure. Use synthetic or explicitly approved non-sensitive materials. Include ambiguity and verification demands that matter to the work, not generic trick questions.
Create an evidence map before launch. Link each decision question to observable evidence: source selection, correction of a plausible error, handling of missing information, use of AI, escalation, revision, final deliverable, and participant feedback. State where the task does not create an opportunity to observe a behaviour.
Dry-run the task with someone outside the design team. Confirm that instructions, materials, timer, tools, support, and submission route work. Check whether the expected evidence can actually be captured and whether the task can be completed in the stated time without relying on hidden knowledge.
### A: Assure access and operation
Run an accessibility and technology check before interpreting results. Test keyboard navigation, screen-reader paths, colour and contrast, zoom, time accommodations, language, document formats, network requirements, and help routes as applicable. Record failures separately from assessment performance.
Calibrate reviewers on anchor examples that include strong, weak, ambiguous, and missing evidence. Require evidence references for dispositions and a route to resolve material disagreement. Track reviewer effort, exceptions, unusable evidence, task defects, and participant questions.
Set operational stop rules. Pause when participants do not receive comparable materials, an access defect affects the evidence opportunity, sensitive data appears, the task answer leaks, reviewer criteria are materially ambiguous, or the platform fails to retain required evidence. Repair the cause before adding more participants.
### L: Lock the exit rule and report
Before launch, define the evidence that supports stop, repair, repeat, or expand cautiously. Avoid a single average threshold. A useful gate combines evidence usefulness, access, task performance, reviewer consistency, operating burden, participant feedback, exceptions, and whether the sponsor can make the stated decision.
Report denominators and flow: invited, eligible, started, completed, assessable, reviewed, and included in each analysis. Show task-bounded patterns, evidence strength, missingness, reviewer disagreement, and limitations. Separate learning exposure, participation, observed performance, and workplace outcomes.
Expansion requires a new boundary. A pilot that works for one synthetic task and cohort does not automatically justify another function, consequence, population, or data environment.
Sources: [1] [2] [3]
Pilot protocol record
| Element | Required entry | | --- | --- | | decision | sponsor, alternatives, date, consequence | | roles | HR, functional, assessment, security/privacy, participant support | | cohort | source population, rationale, invitation, inclusion, exclusions, accommodations | | task | workflow, audience, materials, AI support, time, deliverable, principal failure | | evidence | decision-to-evidence map, capture limits, missingness rules | | operation | dry run, access tests, support, stop rules, incident route | | review | rubric, anchors, calibration, disagreement, reviewer effort | | exit | stop, repair, repeat, expansion conditions | | reporting | audience, denominator flow, limitations, permitted use |
Worked pilot: synthetic procurement-brief review
A fictional organisation wants to decide whether to add targeted verification practice after an AI fundamentals course. Eighteen procurement-operations staff are invited to a development pilot. HR owns participant communication, accommodations, learning follow-up, and the use boundary, so it is an owner for those elements and a collaborator on the overall pilot. The procurement director owns task relevance; the programme sponsor owns the continuation decision.
The 35-minute synthetic task asks participants to review an AI-drafted supplier brief against a request document, a pricing table, and two email clarifications. The brief contains a currency mismatch, an expired deadline, and a plausible unsupported sustainability statement. Participants must submit a corrected brief and an evidence note. No live supplier or personnel data are used.
Sixteen participants start, 15 submit, and 14 submissions are assessable. One participant encounters a keyboard-focus trap in the pricing viewer; that result is excluded from performance interpretation and recorded as an operational defect. Reviewers agree on currency and deadline evidence but disagree on whether a cautious qualifier resolves the unsupported statement. The ambiguity comes from the rubric, not necessarily the participants.
The exit disposition is repair and repeat. The product owner fixes keyboard focus, the assessment owner adds contrasting anchor examples for unsupported claims, and the learning owner adds a short verification exercise. A second bounded case will test whether evidence usefulness and reviewer consistency improve. The pilot does not establish workforce readiness, course impact, productivity, or performance outside this task.
Two Prune boundary
Two Prune can provide a controlled assessment environment, task-bounded workflow evidence, and human-interpreted reporting for a defined pilot. It does not run HR processes, prove learning transfer, establish workplace performance, rank participants for employment decisions, or determine whether a programme should scale.
Sources
UK Government: Employer guide to AI upskilling
- UK Government: AI-upskilling research evidence, analysis and methodology
- NIST: Descriptions of AI actor tasks
Sources: [1] [2] [3]
Sources
- 1.UK employer guide to AI upskilling · UK Government
- 2.UK AI-upskilling research methodology · UK Government
- 3.NIST: Descriptions of AI actor tasks · National Institute of Standards and Technology