Practical guide
How to measure AI adoption without mistaking access for use
Measure AI adoption as a defined work behaviour in a defined opportunity set, not as licences, logins, or self-reported enthusiasm alone. The SCOPE method produces an interpretation-ready measurement card: Specify the construct, establish Coverage, Observe with multiple sources, Probe quality and missingness, and Explain limits. It separates access, attempted use, sustained workflow use, capable.
Measure AI adoption as a defined work behaviour in a defined opportunity set, not as licences, logins, or self-reported enthusiasm alone. The SCOPE method produces an interpretation-ready measurement card: Specify the construct, establish Coverage, Observe with multiple sources, Probe quality and missingness, and Explain limits. It separates access, attempted use, sustained workflow use, capable use, and outcome evidence so one signal is not made to stand for another.
S — Specify the construct and decision
Start with the decision the measure will inform. A programme owner deciding where to improve enablement needs different evidence from a risk owner deciding where human review is weak. Write the construct as an observable behaviour: “eligible team members use an approved AI workflow for a defined task and preserve required verification evidence.”
Define the unit of analysis, population, workflow, observation window, approved tools, and threshold before retrieving data. Keep adjacent constructs separate:
A dashboard may contain all six, but it should not collapse them into one adoption percentage.
- Access: the person can use an approved tool.
- Activation: the person completes an initial meaningful action.
- Attempted workflow use: AI is used on an eligible task at least once.
- Sustained workflow use: repeated use occurs across defined periods or opportunities.
- Capable use: the resulting process and artifact meet stated evidence and review criteria.
- Impact: outcomes differ in a way that a suitable evaluation can attribute to the intervention.
C — Establish coverage and opportunity
Build the denominator from people and tasks that could reasonably exhibit the behaviour. Provisioned accounts are rarely the correct denominator by themselves: some roles have no relevant workflow, some employees were absent, and some tasks did not occur during the window.
Create a coverage table with eligible people, eligible workflow opportunities, observed opportunities, excluded records, and reasons. Stratify only when a subgroup definition is stable and useful. Record changes in role, tool access, workflow design, and system instrumentation because they can create apparent trends without behavioural change.
Coverage should answer three questions: who could have acted, how often the relevant situation occurred, and how much of that opportunity set is visible. If opportunity data are unavailable, say that the measure is user-based and cannot distinguish non-use from no need.
O — Observe with triangulated sources
Choose sources that reveal different parts of the construct and define how they link. Approved-tool events can show interaction; workflow-system events can show eligible cases; artifact review can show source traceability and verification; surveys can reveal barriers; realistic simulations can provide controlled evidence of capability. None is sufficient for every question.
The NIST AI RMF Measure playbook recommends context-relevant metrics, documentation of unmeasured risks, histories or audit logs, and measurement of human oversight, downstream actions, errors, complaints, and responses. For adoption measurement, this means preserving the relationship between a tool event, a work opportunity, a human decision, and the evidence needed to interpret it.
Use privacy-preserving linkage appropriate to the decision. Define join keys, retention, access, aggregation, suppression, and deletion before collection. If individual-level linkage is unnecessary, do not collect it merely because a platform exposes it.
Sources: [1]
P — Probe data quality, missingness, and rival explanations
Run quality checks before calculating the headline measure. Test completeness, uniqueness, consistency, timeliness, validity, linkage success, and representativeness. Compare source totals before and after joins. Inspect whether missingness clusters by role, location, tool, manager, workflow, or date.
The Government Data Quality Framework emphasises fitness for purpose, quality across the lifecycle, clear documentation, anticipation of system changes, and communication of known issues. Apply those principles to the measurement pipeline, not only the final chart.
Maintain a missingness disposition for every material gap:
Check rival explanations such as mandatory training, manager encouragement, a product release, seasonal workload, changed pricing, blocked tools, altered event names, or migration-broken identifiers. A rising event count may reflect more opportunities or new instrumentation rather than deeper adoption.
- exclude under a pre-agreed rule and report the effect on coverage;
- retain as an explicit unknown category;
- repair the source and rerun the release;
- use a bounded estimate with assumptions and sensitivity analysis;
- stop interpretation when the gap undermines the decision.
Sources: [2]
E — Explain the measure and its limits
Publish a measurement card beside every important metric. Include the construct, numerator, denominator, window, source systems, exclusions, linkage rate, refresh date, owner, known gaps, and permitted interpretation. Add a revision log when definitions or instrumentation change.
The UK guidance on evaluating AI interventions advises early evaluation design, a fully specified theory of change, appropriate comparison, iterative evaluation, and transparency about what can be learned at each stage. Adoption telemetry can describe behaviour under its stated definition; it does not by itself establish why behaviour changed or whether the intervention caused an outcome.
Use a release decision for the metric itself: publish, publish with limitation, repair and rerun, or withhold. This prevents a visually polished dashboard from outrunning its evidence.
Sources: [3]
Worked measurement card — support escalation summaries
An 84-person customer-support group introduces an approved workflow for drafting escalation summaries. The programme question is whether eligible staff are sustaining the workflow and preserving source checks after enablement. During six weeks, 72 people are scheduled for escalation duty; the case system records 316 eligible escalations. Tool logs contain 241 draft events, but 19 use an old identifier after a migration and cannot initially link to a case.
SCOPE defines sustained workflow use as at least three linked eligible escalations in two separate weeks, with a submitted summary containing the required source references. The numerator is 46 people; the denominator is 72 eligible people, not all 84 employees or all licensed accounts. Opportunity coverage is 297 of 316 cases after linkage repair. A risk-selected artifact sample finds that 27 of 30 summaries preserve required references; this quality sample is reported separately and is not multiplied into the adoption rate.
The card therefore reports 46 of 72 eligible people meeting the sustained-use definition, 94% opportunity visibility, three sampled artifacts missing required references, and a known manager-assignment imbalance between teams. The decision is publish with limitation and investigate the two low-opportunity teams. The result does not show that AI improved support outcomes, that non-users lack capability, or that the same rate would hold for another workflow.
Two Prune boundary
Two Prune can assess observed performance in realistic simulations and explain the evidence supporting an assessment score. It can contribute a controlled capability signal to a broader adoption measurement system. It does not infer organisation-wide adoption from assessment results, monitor every workplace workflow, certify compliance, or make employment decisions. Customers remain responsible for telemetry governance, denominators, intervention design, and operational interpretation.
Sources
- 1.NIST AI RMF Measure playbook · National Institute of Standards and Technology
- 2.Government Data Quality Framework · UK Government
- 3.UK guidance on evaluating AI interventions · UK Government