Practical guide

AI capability measurement guide for an organizational development leader

An organizational development leader needs an agenda that connects AI capability evidence to work design: changing tasks, decision rights, team interfaces, learning support, and the conditions for a responsible transition. The role may own facilitation and organization-design methods, influence a transformation decision, review workforce effects, or collaborate with functional leaders. Do not.

By Two Prune

An organizational development leader needs an agenda that connects AI capability evidence to work design: changing tasks, decision rights, team interfaces, learning support, and the conditions for a responsible transition. The role may own facilitation and organization-design methods, influence a transformation decision, review workforce effects, or collaborate with functional leaders. Do not assume the title owns technology, policy, budget, or every operating decision.

Define the decision this agenda must produce

Begin with one workflow and one decision due at the end of the meeting. For example: “Should the customer-operations team test a redesigned complaint-triage workflow, and under what conditions?” Name the executive decision owner, workflow owner, organizational development lead, technology owner, risk reviewers, affected workers, and evidence owner.

Bring four alternatives into the room:

The OECD’s 2026 AI-ready public workforce brief links capability building with institutional strategy, governance, multidisciplinary skill, experimentation, continuous learning, and changes in work organization. Its public-sector examples are useful design evidence, not a universal mandate for private organizations. Building an AI-ready public workforce

  • keep the current workflow;
  • improve the process without AI;
  • run a controlled AI-assisted design; or
  • defer while a material evidence gap is closed.

Sources: [1]

Use a five-decision agenda

This is not a presentation sequence. Each segment must close with an explicit decision, owner, evidence requirement, or unresolved question.

### Decision 1: What work is changing?

Map the current task, user or customer outcome, inputs, source authority, handoffs, queues, exceptions, controls, and accountable owner. Then identify what the AI-assisted design would remove, reduce, transform, or create.

Ask where invisible work moves. Faster drafting may create more source checking; automated triage may create exception review; an assistant may shift explanation work to a frontline team. Record both the proposed future state and the transition work needed to reach it.

Output: a one-page task-change ledger with current actor, proposed actor, AI contribution, retained human judgment, new work, removed work, and affected handoff.

### Decision 2: Which decision rights and safeguards must remain explicit?

For every consequential step, name who performs it, who is accountable, who can override it, and who can stop the workflow. Separate tool operation from decision authority. Include policy, privacy, security, accessibility, employee representation, professional judgment, and customer recourse where relevant.

NIST’s AI Risk Management Framework calls for clearly defined roles, operator proficiency, human oversight, documentation, and monitoring. These are useful agenda prompts; framework alignment alone does not show that a proposed organization design will work. NIST AI RMF Core

Output: a decision-rights table and an escalation route for normal, uncertain, and prohibited cases.

### Decision 3: What capability and support does the work system require?

Translate changed work into observable capability: interpreting the task, finding evidence, using AI within allowed boundaries, checking claims or calculations, recognizing uncertainty, escalating, and communicating the result. Then name the conditions people need to demonstrate it—time, tools, access, practice, feedback, accessible materials, manager support, and review capacity.

The UK government’s 2026 employer guide describes effective AI upskilling as practical, reachable, integrated, modular, expandable, and sustainable. It emphasizes real tasks, accessible participation, governance, follow-up, and continued review. Use that evidence to test the learning system, while measuring transfer in the organization’s own context. Employer guide: What works for AI upskilling in the UK

Output: a capability-and-conditions matrix, not a generic course list.

### Decision 4: What transition can be tested without hiding adverse effects?

Choose a bounded workflow, representative cases, comparison, duration, support model, and stop conditions. Include hard cases and exceptions, not only the happy path. Collect workload shifts, review effort, waiting, error catches, workarounds, overrides, accessibility barriers, and participant feedback alongside output quality.

Do not describe a pilot as successful merely because people completed it or used the tool. Define what evidence would support expanding, repairing, stopping, or reverting the design.

Output: a transition experiment charter with owners, sample, comparison, evidence plan, pause authority, and exit rules.

### Decision 5: What evidence will justify the next organization-design move?

Keep assessment evidence, workflow evidence, workforce experience, and downstream outcomes separate. State denominators, missingness, confounds, and transfer limits. Decide who receives the report and what each recipient can legitimately conclude.

Output: a decision gate with four possible dispositions: continue current design, repair and retest, expand within a stated boundary, or stop.

Sources: [2] [3]

Worked agenda: complaint-triage redesign

A synthetic customer-operations team is considering an assistant that summarizes complaints and proposes a category. The current process has inconsistent labels and long waits, but specialists often detect product-safety cues in free text.

| Agenda decision | Evidence brought to the meeting | Decision recorded | | --- | --- | --- | | Work change | Ten current cases, queue data, source policy, specialist interviews | AI may draft summaries; specialists retain category and escalation decisions | | Decision rights | Current escalation policy and three ambiguous examples | Safety-related cues always route to a named specialist; the operations manager can pause the pilot | | Capability and support | Synthetic assessment with conflicting product notes and incomplete complaints | Participants need source-navigation, uncertainty, and escalation practice; managers need review capacity | | Transition test | Current-process baseline and 30 representative synthetic cases | Run a two-week shadow test; no AI-proposed category reaches a customer record | | Evidence gate | Accuracy by category, missed escalation, review time, rework, feedback, missing cases | Expand only if every safety cue is escalated and review burden remains supportable; otherwise repair or stop |

The organizational development leader owns the work-design session and capability matrix, influences the transition choice, and reviews role and support effects. The customer-operations director owns the operating decision. Product safety owns escalation criteria. Technology owns tool configuration. This classification prevents “organizational development” from becoming an unnamed owner of every risk.

Objections to resolve in advance

“Can we use adoption data instead?” Adoption shows activity, not whether people can perform the changed work or whether the new handoff is viable.

“Why not train everyone first?” Training before the work and decision rights are defined can prepare people for the wrong task. Design the bounded workflow and capability requirement together.

“Does an assessment prove the redesign works?” No. It can reveal observed capability in a controlled simulation. Live workflow performance, workforce effects, and customer outcomes require separate evidence.

“Can one pilot settle the organization design?” A pilot can reduce a defined uncertainty. It cannot establish transfer across functions, tools, volumes, or consequences that were not tested.

Reporting boundary

Report the exact workflow, task version, participant opportunity, support conditions, evidence collected, missing cases, adverse findings, and unresolved risks. Avoid a single transformation score. Preserve prohibited or consequential failures separately from averages. Name whether the organizational development leader is owner, influencer, reviewer, or collaborator for the next decision.

Two Prune can supply evidence from realistic AI-enabled work simulations, including source use, AI interaction, verification, revision, escalation, and the final deliverable. It does not redesign the customer’s organization, prove operational impact, or make an employment decision.

Sources and limits

The worked agenda is synthetic. The sources inform the decision questions but do not establish a single correct organizational design or a Two Prune customer outcome.

  • OECD: Building an AI-ready public workforce — workforce capability, governance, multidisciplinary skill, experimentation, and continuous learning.
  • GOV.UK: Employer guide — What works for AI upskilling in the UK — current evidence and PRIMES principles for practical and sustainable AI learning.
  • NIST AI RMF Core — roles, proficiency, oversight, documentation, and monitoring considerations.

Sources: [1] [3] [2]

Sources

  1. 1.Building an AI-ready public workforce · www.oecd.org
  2. 2.NIST AI RMF Core · National Institute of Standards and Technology
  3. 3.Employer guide: What works for AI upskilling in the UK · UK Government