Practical guide
How to report AI capability evidence to a functional practice leader
Report AI capability evidence to a functional practice leader as a practice decision packet, not a leaderboard or an activity dashboard. The leader needs to see what work standard was tested, what participants had the opportunity to demonstrate, what evidence was actually available, which patterns may warrant practice changes, and where interpretation must stop. The report should connect.
Report AI capability evidence to a functional practice leader as a practice decision packet, not a leaderboard or an activity dashboard. The leader needs to see what work standard was tested, what participants had the opportunity to demonstrate, what evidence was actually available, which patterns may warrant practice changes, and where interpretation must stop. The report should connect evidence to an owner and next action while keeping individual assessment evidence within the approved access boundary.
Start with the leader’s decision and role
Classify the functional practice leader as the owner of a work standard, an influencer on investment, a reviewer of practice evidence, or a collaborator with learning, technology, risk, and business leaders. They may hold several roles, but do not assume they control budget, employment decisions, system deployment, or participant management.
Name the decision before selecting metrics. Examples include whether to revise a verification standard, commission targeted practice, repair a workflow, investigate shifted review burden, or repeat a bounded evidence pilot. If the result cannot inform a named decision, a short evidence note may be more appropriate than a formal report.
UK Cabinet Office guidance says functional standards can provide a stable basis for proportionate assurance, risk management, capability improvement, role clarity, and continuous improvement. It applies to UK government functions, so use it as an authoritative example of reporting against defined standards rather than as a universal organizational model. UK Cabinet Office: Guide to functional standards
Sources: [1]
Build the SIGNAL packet
SIGNAL keeps group reporting tied to evidence rather than turning limited observations into broad claims.
### S: Standard and scope
State the observable practice standard, task, participant inclusion rule, time window, scenario version, materials, available tools, and expected deliverable. Explain which behaviors the task created an opportunity to demonstrate. Separate the assessment boundary from live workplace performance.
Include the reporting audience and access rule. A practice leader may need an aggregate pattern and de-identified examples, while a participant or manager may have a different approved report. Do not expose individual evidence merely because it is technically available.
### I: Inputs and integrity
List the evidence inputs: submitted deliverables, source use, available workflow events, AI interactions where captured, reviewer annotations, scoring records, missing evidence, and scenario metadata. State which inputs were unavailable or unreliable.
Record the assessment and rubric versions, reviewer protocol, calibration process, and any changed rule. Distinguish evidence-location agreement from judgment agreement. A score without lineage to the work standard and supporting evidence is not sufficient for a practice decision.
### G: Group patterns with denominators
Present counts and denominators before percentages. Separate observed behavior, reviewer interpretation, and hypothesis. Show how many participants had a valid opportunity to demonstrate each behavior, how many records were interpretable, and how many were missing or excluded under predeclared rules.
Avoid one composite average when it hides material patterns. A group may communicate clearly while repeatedly missing source contradictions. Report verification, evidence navigation, judgment, and communication patterns separately where the task supports them. Small groups and selected volunteers limit generalization.
### N: Nuance, alternatives, and uncertainty
Include plausible alternative explanations such as task ambiguity, unfamiliar interface, source quality, accessibility barriers, time pressure, reviewer drift, work-experience differences, or an unrepresentative sample. State what the report cannot establish about causation, workplace transfer, productivity, safety, or future performance.
The Cabinet Office’s audit and assurance guide emphasizes proportionate use, defined assurance approaches, evidence-based information, and checking whether arrangements are being followed and having their intended effect. It does not make a self-assessment or report a substitute for the underlying standard. Using functional standards for audit and assurance
### A: Actions and owners
Connect each supported pattern to a proportionate option: retain the current standard, clarify an instruction, repair source access, design practice, investigate live-work evidence, retrain reviewers, repeat a bounded assessment, or stop a weak measurement. Name the practice owner, collaborator, evidence needed, and due date.
Do not turn a development pattern into an individual employment recommendation. The report can identify practice questions and follow-up evidence needs for human interpretation.
### L: Learning gate and expiry
Finish with the decision requested, the evidence threshold for that decision, and the date the packet expires. A new scenario, participant population, rubric, model, source set, or workflow context may make the earlier result non-comparable. Keep the frozen report and issue an addendum rather than silently rewriting it.
Sources: [2]
Copyable reporting matrix
| Practice question | Opportunity in task | Available evidence | Pattern and denominator | Missingness or alternative | Proposed action | Owner | Expiry | | --- | --- | --- | --- | --- | --- | --- | --- | | Are source conflicts being escalated? | one conflicting policy pair in every case | deliverable, source openings, reviewer trace | count / valid opportunities | off-screen checks unavailable | targeted practice or scenario repair | named practice owner | date |
Attach a scope card, evidence lineage, pattern table, selected de-identified examples, uncertainty ledger, reviewer-disagreement note, action register, and result-use restrictions. Put individual assessment reports behind their own approved access route.
Worked example: proposal-quality evidence
A functional practice leader receives a slide saying “verification capability averaged 68%” after twelve volunteers complete a synthetic proposal red-team task. The slide does not identify the standard, denominator, evidence gaps, or action.
Standard and scope: the revised packet states that participants had 30 minutes to identify mandatory requirements, trace corrections to supplied sources, flag unsupported statements, and communicate a prioritized correction plan. It covers one synthetic scenario and does not establish live-bid performance.
Inputs: all twelve deliverables are available. Workflow events are incomplete for two sessions, and one participant used an approved accommodation. Reviewers used a frozen protocol and double-reviewed four submissions. The accommodation is not treated as a defect, and missing events do not become negative evidence.
Group patterns: ten of twelve participants encountered the seeded source conflict in an interpretable way; six escalated it, three chose one source without explaining the conflict, and one produced insufficient evidence for interpretation. Eleven of twelve had an interpretable opportunity to check a plausible unsupported claim; eight flagged it. The packet shows these counts rather than a composite ranking.
Nuance: the source-conflict instruction may have been less visible than intended, and the volunteers were drawn from one experienced team. The pattern supports a practice question but not a causal claim about AI use or the wider function.
Action: the practice leader owns a two-part response: repair and retest the scenario cue with three reviewers, then run a short source-conflict practice exercise for the participating team. A learning partner collaborates; the scenario owner repairs the task. The report expires after 60 days or any material scenario change.
The disposition is targeted practice plus measurement repair, not “low capability.” The packet gives the leader an actionable standard, denominator, uncertainty, owner, and next evidence gate without ranking people.
Reporting failure conditions
Hold the report when the standard changed after results were visible, denominators are unavailable, a material group lacked a fair opportunity, missing evidence is treated as failure, examples expose restricted participant information, reviewer disagreement is concealed, or the requested action exceeds the evidence. Repair a mislabeled or ambiguous task before interpreting it as a participant pattern.
Two Prune produces task-bounded assessment evidence for human interpretation. It does not prove workplace performance, promise practice outcomes, rank participants, or make automated employment decisions.
Sources
UK Cabinet Office: Guide to functional standards
- UK Cabinet Office: Using functional standards for audit and assurance
Sources: [1] [2]
Sources
- 1.UK Cabinet Office: Guide to functional standards · UK Government
- 2.Using functional standards for audit and assurance · UK Government