Practical guide
AI capability measurement guide for the digital transformation leader
A digital transformation leader needs an AI capability agenda that connects service outcomes, workflow changes, workforce evidence, technology dependencies, governance, and investment decisions. Tool adoption alone cannot show whether people can use AI effectively in real work, whether controls operate, or whether the transformation is improving the service.
A digital transformation leader needs an AI capability agenda that connects service outcomes, workflow changes, workforce evidence, technology dependencies, governance, and investment decisions. Tool adoption alone cannot show whether people can use AI effectively in real work, whether controls operate, or whether the transformation is improving the service.
Classify the leader’s role for each decision
Treat the title as context, not authority. A digital transformation leader may own the portfolio agenda, influence business-unit priorities, review platform evidence, or collaborate with learning, security, data, risk, finance, and service owners. Name the actual decision owner for funding, production release, workforce development, data use, and operational risk.
The UK Government Digital, Data and Technology Playbook frames digital delivery around outcomes, multidisciplinary capability, user needs, iterative delivery, commercial planning, and lifecycle management. The OECD Digital Government Policy Framework emphasizes digital-by-design government, data-driven decisions, government as a platform, openness, user-driven services, and proactiveness. These sources support a portfolio agenda; they do not prove that a specific AI initiative or assessment produces transformation outcomes. UK Government: Digital, Data and Technology Playbook OECD: Digital Government Policy Framework
Sources: [1] [2]
Use the SHIFT agenda
### S — Service outcome, scope, and authority
Start with a service or operating outcome, affected users, current baseline, decision date, and accountable owner. Define which workflows, populations, systems, and jurisdictions are included. State the consequence of maintaining the current approach.
Separate transformation outcomes from activity metrics. Examples of activity are licenses, logins, training completion, prompts, and prototypes. Examples of service evidence are quality, error patterns, rework, time to resolution, accessibility, incidents, user experience, and control operation. Neither category alone proves causation.
### H — Hypotheses and alternatives
Write a testable hypothesis for each proposed change: what workflow constraint changes, for whom, under which conditions, and what evidence would challenge it. Compare at least no change, process redesign without AI, targeted support or learning, a bounded AI test, and broader implementation where appropriate.
Include non-capability explanations such as poor access, unclear policy, weak data, interface friction, missing managerial support, unsuitable workflow, or incentives. A capability programme should not become the default answer to every adoption gap.
### I — Interdependencies and capability evidence
Map user needs, work steps, source data, technology, integrations, controls, accessibility, commercial dependencies, operating owners, and fallback routes. For workforce evidence, distinguish:
| Evidence layer | Example | What it can inform | What it cannot establish alone | | --- | --- | --- | --- | | Access | approved tool and data availability | feasibility barrier | capability | | Participation | starts, completion, support use | reach and burden | task performance | | Knowledge | scenario or policy check | conceptual understanding | workplace behavior | | Simulated work | process and deliverable evidence | performance under stated conditions | transfer | | Work sample | reviewed operational artifact | behavior in sampled work | organization-wide effect | | Service outcome | quality, rework, time, incidents | operational pattern | attribution without design |
Require evidence lineage, task versions, missingness, review method, disagreement, and interpretation limits. Missing evidence is not poor performance.
### F — Funding, sequencing, and portfolio gates
Sequence dependencies before scaling. A portfolio card should identify the decision, owner, baseline, alternative, hypothesis, evidence, cost range, capacity, control prerequisites, pilot design, exit thresholds, and next review.
Use gates such as discover, design, test, repair, expand, hold, stop, and retire. Funding should follow evidence needs and reversible stages rather than treating sunk cost or executive enthusiasm as proof. Name shared platform work separately from workflow-specific benefits so one programme does not claim all value.
### T — Thresholds, reporting, and terminal decisions
Pre-register thresholds for feasibility, decision usefulness, operating burden, safeguards, and evidence quality. Report distributions and failure patterns rather than a single transformation score. Preserve alternatives, dissent, exceptions, and uncertainty.
Terminal decisions should be explicit: maintain, revise workflow, change enablement, run a transfer test, expand the sample, approve a bounded implementation, hold for missing authority, stop, or retire. Each needs an owner, evidence packet, and date.
Decision agenda
Use these recurring questions:
- Which service outcome and user problem have priority, and who owns them?
- Is the constraint capability, access, process, data, technology, policy, management, or several together?
- What alternatives could solve the problem without the proposed AI change?
- What realistic work evidence is needed before a funding or scaling decision?
- Which data, platform, commercial, accessibility, security, and governance dependencies must be resolved?
- What evidence would falsify the current hypothesis?
- Which conditions trigger repair, expansion, hold, stop, or retirement?
- How will operating and capability evidence be reported without overstating transfer or causation?
Worked agenda: service-request triage
A synthetic organization proposes an AI assistant for internal service-request triage. The transformation dashboard currently shows license activation and training completion. Leaders want to scale because adoption exceeds 80%.
SHIFT reframes the agenda around correct routing, time to acknowledged ownership, avoidable reassignment, accessibility, and incident risk. The transformation leader is portfolio owner and collaborator; the service owner owns the operational decision, security owns data controls, procurement owns vendor terms, and learning owns support design.
The alternatives are improve the request form and taxonomy, add human triage guidance, run a bounded assistant test, or deploy broadly. Workflow evidence shows that a large share of reassignment comes from overlapping service categories, so capability is only one hypothesis.
The bounded test uses synthetic and approved historical requests, includes ambiguous categories and conflicting clues, and records source review, AI interactions, human corrections, final routing, confidence, and escalation. Reviewers examine routing quality, unsupported inference, correction burden, accessibility issues, and whether the fallback works.
The synthetic result shows faster initial suggestions but repeated confusion between access incidents and account provisioning. Experienced users correct the issue; newer users accept more incorrect suggestions. The operating team also finds that the source taxonomy changed during the test.
The portfolio decision is repair before expansion: fix the taxonomy, add a visible source-date warning, provide escalation practice, and repeat with a frozen version. License use and course completion remain context, not outcome evidence. The agenda does not claim productivity, return on investment, workforce transfer, or organization-wide readiness.
The evidence packet contains the outcome definition, authority map, alternatives, workflow dependency map, capability evidence design, operating results, reviewer disagreement, support burden, safeguards, decision, owners, thresholds, and next review date.
Objections and failure conditions
Answer “we already have usage data” by showing the distinct decision that requires work evidence. Answer “managers can observe capability” by defining and comparing the observation method. Answer “the pilot is artificial” by stating represented conditions and the separate transfer gate. Answer “measurement slows delivery” by removing evidence that does not inform a decision, not by removing verification.
Hold the agenda when outcomes are undefined, authority is assumed, tool adoption substitutes for capability, alternatives are missing, evidence lineage cannot be reproduced, small groups are overinterpreted, controls lack owners, or scaling thresholds are invented after results.
Two Prune provides realistic AI-enabled work assessments and evidence about observed task performance. It can contribute one evidence layer to a transformation portfolio; it does not run the transformation, prove business impact, certify readiness, or make employment decisions.
Sources
UK Government: Digital, Data and Technology Playbook
- OECD: The OECD Digital Government Policy Framework
Sources: [1] [2]
Sources
- 1.UK Government: Digital, Data and Technology Playbook · UK Government
- 2.OECD: Digital Government Policy Framework · www.oecd.org