Practical guide
How to report AI capability evidence to a digital transformation leader
Report AI capability evidence to a digital transformation leader as a decision packet, not a score dashboard. Connect observed work evidence to a defined transformation decision, show adoption and capability as different constructs, expose missingness and alternative explanations, classify stakeholder authority, and end with an owned next evidence gate. A report should make it easier to choose.
Report AI capability evidence to a digital transformation leader as a decision packet, not a score dashboard. Connect observed work evidence to a defined transformation decision, show adoption and capability as different constructs, expose missingness and alternative explanations, classify stakeholder authority, and end with an owned next evidence gate. A report should make it easier to choose an action without claiming that the evidence proves transfer or business impact.
Mandate and source boundary
Classify the transformation leader as owner, influencer, reviewer, or collaborator for each action. Service owners may own workflow outcomes; learning may own support; security, privacy, and accessibility functions own their controls; finance and procurement own investment and contracts. Do not infer authority from the title.
The UK Government Digital, Data and Technology Playbook emphasizes measurable outcomes, user needs, iterative delivery, testing, governance, and breakpoints before scale. The Department for Business and Trade’s digital evaluation strategy integrates performance analysis and evaluation into digital delivery. They support decision-oriented reporting but do not establish that an assessment causes operational improvement. UK Government: Digital, Data and Technology Playbook Department for Business and Trade: Digital evaluation and performance analysis strategy
Sources: [1] [2]
Report with SIGNAL
### S — Scope, stakeholder, and specific decision
Name the decision, decision owner, date, affected workflow, population, evidence period, and available options. State why the reader needs the report now.
Distinguish a portfolio update, pilot breakpoint, support decision, workflow redesign, broader test, investment request, and production release. One report can inform several stakeholders, but each action needs its own authority.
### I — Indicators in separate evidence layers
Organize indicators by what they can show:
Never use an easier layer as a substitute for a harder one. High activation is not capability evidence. Strong task performance is not transfer or service impact.
For every indicator, record definition, source, population, period, denominator, missingness, owner, and interpretation boundary.
### G — Gaps, groups, and competing explanations
Show who had the opportunity to generate evidence and who did not. Separate non-participation, technical failure, accommodation, missing telemetry, incomplete submission, and valid abstention from poor performance.
Report relevant context only when sample size, consent, privacy, and interpretation permit it. Small-group results may need suppression or narrative treatment. Do not rank groups when task exposure or evidence opportunity differs.
List competing explanations. Low task performance could reflect unclear materials, unfamiliar tools, accessibility barriers, taxonomy problems, time limits, or capability. High usage could reflect mandate rather than value.
### N — Narrative, numbers, and evidence strength
Lead with the decision and observed pattern. Follow each material claim with its evidence, strength, and limitation. Use counts and distributions before percentages when populations are small.
Show the full score or measure context: construct, task, rubric, evidence sources, reviewer design, missingness, uncertainty, and calibration state. Label directional benchmarks as directional and avoid precision unsupported by the reference population.
Include representative evidence patterns without exposing personal or confidential information. State where reviewers disagreed and how adjudication affected interpretation.
### A — Actions, alternatives, and breakpoint
Offer action alternatives tied to the evidence: no change, repair access, improve guidance, redesign workflow, target practice, revise the assessment, collect transfer evidence, investigate another cause, extend the pilot, scale, restrict, or stop.
For each, name owner, cost or burden range, dependency, expected learning, risk, date, and stop condition. Separate evidence-supported action from hypothesis.
### L — Limits, lineage, and next evidence gate
Close with what the report cannot establish, the exact source and version lineage, residual uncertainty, and evidence that would change the decision. Link the next gate to a date or trigger.
Retain the decision charter, population and opportunity record, metric dictionary, task version, evidence manifest, scoring and review record, findings, exceptions, report version, decision, and follow-up.
- Access: approved tool, account, permissions, equipment, support.
- Activity: activation, frequency, feature or workflow use.
- Learning: attendance, completion, practice, self-report.
- Observed capability: performance and process in a bounded realistic task.
- Transfer: evidence from approved workplace samples or observation.
- Service outcome: quality, delay, rework, incident, user or cost measures.
Decision packet
| Section | Required content | Reader question | | --- | --- | --- | | Decision | owner, options, deadline | What must be chosen? | | Evidence layers | access through outcomes | What was actually observed? | | Opportunity and missingness | who could produce evidence | Who is absent and why? | | Findings | strength, contradiction, uncertainty | What does the record support? | | Alternatives | burden, risk, reversibility | What else could we do? | | Action | owner, threshold, stop trigger | What happens next? |
Worked report: AI-supported procurement briefs
A synthetic transformation programme has 120 staff with approved AI access. Leadership wants to report “readiness” after a small capability pilot.
SIGNAL reframes the decision: should the programme repair the pilot and run a broader test, move directly to workflow deployment, or stop additional measurement? The transformation leader owns the pilot breakpoint but not procurement policy, privacy approval, or production release.
The report shows 112 active accounts, 89 monthly users, 76 course completions, 18 pilot invitations, and 17 completed synthetic assessments. These are four different populations. It does not convert them into one readiness percentage.
Observed task evidence shows that most participants found the governing requirement, while five accepted an unsupported supplier statement and four missed a cost-period mismatch. Reviewers disagreed on two escalation judgments. One participant used an accommodation; two event records lack source-close events. The report treats missing telemetry as a capture limitation.
Competing explanations include small self-selected participation, different procurement experience, spreadsheet accessibility at high zoom, and an ambiguous instruction about escalation. The report therefore does not generalize the pattern to all 120 staff.
Alternatives are: deploy now; repair the task and telemetry; add a source-verification practice module; sample approved live work; or combine repair with a broader stratified assessment. Deployment lacks transfer and service evidence. Live work contains sensitive supplier data and requires a separate design.
The recommendation is repair accessibility and event capture, clarify escalation, then invite a representative second cohort. Thresholds require complete technical evidence, acceptable reviewer agreement, documented evidence opportunity, and an interpretable pattern before any transfer study. Procurement retains authority over workflow changes.
The packet contains the authority map, metric dictionary, population funnel, task and rubric versions, evidence completeness report, distributions, reviewer disagreement, pattern examples, alternative table, decision, and next gate. It makes no workforce-wide readiness, productivity, ROI, or employment claim.
Reader objections
“Why not show one score?” A single number can hide evidence layers, opportunity, missingness, and consequential error patterns. Use it only with the construct and limitations visible.
“Why report activity if it does not show capability?” Activity can diagnose access and adoption. It remains useful when its boundary is explicit.
“Can this prove the transformation works?” No. The report can inform a bounded capability or measurement decision. Transfer and service outcomes need separate evidence.
Hold publication when populations are conflated, missing evidence becomes low performance, a directional benchmark is presented as mature, stakeholder authority is wrong, subgroup privacy is weak, or the recommended action exceeds the evidence.
Two Prune provides realistic AI-enabled work assessments, scores observed performance, and explains supporting evidence. It can supply one evidence layer to a transformation report. It does not own transformation outcomes, validate business impact, or make employment decisions.
Sources
UK Government: Digital, Data and Technology Playbook
- Department for Business and Trade: Digital evaluation and performance analysis strategy
Sources: [1] [2]
Sources
- 1.UK Government: Digital, Data and Technology Playbook · UK Government
- 2.Department for Business and Trade: Digital evaluation and performance analysis strategy · UK Government