Measuring AI skills gaps in knowledge work

Anchor the gap to a real task

Describe the work product, audience, sources, permitted AI contribution, material failure modes, and verification standard. Translate these into observable behaviors such as framing a decision, selecting an authoritative source, decomposing a task, challenging an unsupported output, reconciling a conflict, or communicating uncertainty.

Do not infer a gap when the exercise offered no fair opportunity to demonstrate the behavior. Missing system access, inaccessible materials, unclear instructions, time pressure, prior domain knowledge, language, and reviewer inconsistency can all affect the evidence.

The UK government's AI foundation skills for work benchmark describes foundational technical and non-technical skills for workers. It can inform a skills vocabulary, but a local organization still needs to connect that vocabulary to representative tasks and operating conditions.

Sources: [1]

Build the opportunity-to-evidence gap map

Create one row per required behavior. Record the task moment, expected behavior, observable evidence, opportunity present or absent, observed result, alternative explanation, confidence, work-environment dependency, learning response, reassessment method, and owner.

Use dispositions such as demonstrated in this task, partial evidence, not demonstrated despite opportunity, and not observable. Keep confidence separate from performance. One missed check in one scenario can identify a training hypothesis without establishing a stable trait.

Define the evidence anchor before reviewing attempts. For verification, an anchor might require locating the controlling source, checking the material value, and correcting or escalating the output. Record reviewer disagreement and revise ambiguous anchors before comparing groups.

The OECD report Building an AI-ready public workforce connects workforce capability with leadership, governance, strategic alignment, and continuous learning. Its public-sector evidence helps frame organizational dependencies but does not establish a universal benchmark for individual workers.

Sources: [2]

Worked artifact: a synthetic customer escalation

Imagine a synthetic customer-operations exercise. A participant must review an account history, a service policy, a disputed invoice, and an AI-drafted response. The task requires source selection, conflict resolution, escalation judgment, verification of amounts, and clear communication.

The draft response cites the current service policy but repeats a refund amount from an older invoice. The event record shows that the participant opened both invoices, asked AI to compare them, and edited the final response without recalculating the amount. The map marks verification as not demonstrated despite a visible opportunity, with moderate confidence because the exercise captured the source trail and revision.

The participant never encounters conflicting policy versions, so evidence about authority resolution is marked not observable—not a deficit. A slow source viewer also consumes part of the task time, and that environmental constraint is retained. The learning action is a short practice on claim-to-source tracing and recalculation, followed by a different synthetic case.

The follow-up can show whether the behavior appears under comparable conditions. It cannot prove durable transfer to every workflow. The people, accounts, and amounts are synthetic, and the result is for development and human interpretation rather than an automatic employment decision.

Aggregate gaps without losing context

At team level, count evidence opportunities before counting misses. Group by task, behavior, workflow condition, and confidence. A cluster of verification misses may point to learning needs, poor interfaces, missing source authority, or unrealistic workload. Preserve those explanations instead of ranking people on a single composite number.

Review rater agreement, scenario equivalence, accessibility, missing telemetry, and changes in tools or policy. Refresh the map when the task changes. Two Prune can supply structured evidence from bounded simulations; it does not establish universal proficiency, future job performance, or a mature population benchmark.