Practical guide
How to report AI capability evidence to a future-of-work researcher
A future-of-work researcher usually needs interpretable evidence about changing tasks, capability, working conditions, and uncertainty. They are commonly an influencer or reviewer, not automatically the budget owner or programme operator. Report what the study can support, how the evidence was produced, where it is missing, and which decision the findings can inform. Use SIGNAL: Scope,.
A future-of-work researcher usually needs interpretable evidence about changing tasks, capability, working conditions, and uncertainty. They are commonly an influencer or reviewer, not automatically the budget owner or programme operator. Report what the study can support, how the evidence was produced, where it is missing, and which decision the findings can inform. Use SIGNAL: Scope, Indicators, Groups, Narrative, Action, and Limits. The result is a research brief with a reproducible evidence map, not a dashboard of AI activity or a promotional score summary.
Classify the stakeholder and decision
Confirm the researcher's mandate before preparing the report:
Name the actual decision owner separately. A researcher may advise an AI enablement director on whether a pilot should continue, help a workforce council interpret task changes, or review a learning study. Do not assume they authorize procurement, training, or workforce policy.
Write the decision question, for example: “Does this bounded assessment provide enough evidence to test a source-verification practice module in two operations teams?” That is materially different from “Are employees ready for AI?”
- Owner: accountable for the research decision and resources.
- Influencer: shapes methods, interpretation, or recommendations.
- Reviewer: challenges evidence quality and reporting.
- Collaborator: contributes domain, data, or implementation knowledge.
S: Freeze scope and construct
Define the reporting boundary before collecting or comparing evidence:
The OECD's review of AI measurement in ICT usage surveys shows that definitions, scope, coverage, and survey design affect comparability. A report should therefore state exactly what counts as AI use and what the instrument measures. OECD: AI measurement in ICT usage surveys
For capability evidence, define behaviors rather than attitudes. “Verification and risk control” might mean checking source authority, reconciling conflict, testing calculations, marking uncertainty, and escalating when evidence is inadequate. Confidence or frequency of use can be useful context but is not a substitute for observed behavior.
- population and recruitment method;
- work context, task family, and period;
- AI tools and versions available;
- capability construct and excluded interpretations;
- evidence captured and known gaps;
- intended decision and prohibited uses.
Sources: [1]
I: Build an indicator dictionary
For every reported indicator, provide:
| Field | Example | | --- | --- | | name | supported source-reconciliation decision | | construct | verification behavior in a policy case | | numerator | submissions that identify and resolve the controlling conflict | | denominator | complete, reviewable sessions exposed to the conflict | | source | frozen deliverable, source trail, and reviewer record | | missingness | incomplete capture reported separately | | segmentation | team and prior task familiarity, where sample permits | | decision link | whether to test targeted practice | | limitation | one synthetic task does not establish everyday transfer |
Do not mix activity and capability in one rate. Opening a document, sending prompts, or spending more time may describe a process, but none is inherently better. Interpret process signals against the task, expected method, output, and reviewer rationale.
G: Preserve groups, denominators, and collection differences
Show who is represented and who is absent. Report invitations, participation, exclusions, completed sessions, incomplete sessions, and reviewable sessions. Explain voluntary participation, selection effects, tool access, accommodations, and any data-collection changes.
The OECD's employer and worker surveys deliberately use complementary perspectives: employers describe implementation while workers describe first-person experience. That design is a reminder to avoid treating one vantage point as the whole workplace. OECD: AI surveys of employers and workers
When external labour statistics appear in the report, keep their definitions and provenance. ILOSTAT notes that international statistical standards, source processing, and harmonization affect interpretation and comparability. Use official data as context, not as a benchmark for a local simulation unless the construct, population, and method genuinely align. ILOSTAT: Statistical standards and methods
Sources: [2] [3]
N: Write a layered evidence narrative
Structure the report in four layers:
Place an evidence map beside the narrative:
| Claim | Evidence | Strength | Counterevidence or confound | Decision use | | --- | --- | --- | --- | --- | | source conflict creates difficulty in this task | 14 of 31 reviewable sessions miss or mishandle the later exception | moderate for this sample | unfamiliar interface may contribute | design a targeted follow-up | | participants use AI frequently | interaction present in 27 sessions | descriptive | offline use not captured | operating context only | | training will improve verification | none | unsupported | no intervention comparison | do not report |
- Observed: counts, artifacts, reviewer dispositions, missingness, and operational burden.
- Interpreted: what the frozen construct and rubric allow the team to infer.
- Alternative explanations: selection, task familiarity, source design, reviewer drift, tool access, or capture failure.
- Unknown: transfer, persistence, causal effect, and unrepresented groups.
A: Connect findings to a bounded action
A useful recommendation specifies owner, action, evidence gate, and review date. Examples:
The U.S. GAO AI Accountability Framework organizes questions around governance, data, performance, and monitoring. It is designed for federal agencies and other entities, so present its authority accurately. Its questions can help a researcher check whether roles, data quality, metrics, monitoring, and corrective actions are visible in the report. U.S. GAO: AI Accountability Framework
- repair capture before repeating the study;
- run alternate task forms to test task-specificity;
- calibrate reviewers on conflicting-source anchors;
- add worker interviews to understand operating constraints;
- test a targeted practice module with a pre-specified comparison;
- hold expansion because the sample excludes a materially affected group.
Sources: [4]
L: State limits where readers make decisions
Put limitations beside the finding they qualify, not only in an appendix. State whether evidence is descriptive, comparative, diagnostic, or exploratory. Avoid causal language unless the design supports it. Avoid precise population estimates from a convenience sample. Never convert missing process evidence into a low capability score.
Record model, task, rubric, and source versions. A report can become stale when the workflow, policy, AI tool, or workforce changes.
Worked reporting brief
Decision. The AI enablement director must decide whether to test a source-verification practice module. A future-of-work researcher is the method owner and influential reviewer. Operations supplies task expertise, learning owns any intervention, and governance reviews data use.
Scope. Forty volunteers from two customer-operations teams receive a 45-minute synthetic policy case. The case contains a general rule, a later exception, a spreadsheet, and an inbox clarification. Participants may use an approved assistant and must submit a decision memo with source anchors. Thirty-four complete the task; three sessions have critical capture loss, leaving 31 reviewable sessions.
Observed evidence. Fourteen of 31 reviewable submissions miss or mishandle the later exception. Ten reconcile the conflict and explain the controlling source. Seven identify the conflict but provide an incomplete rationale. AI interaction is present in 27 sessions, but both supported and unsupported submissions include AI use. Reviewers disagree on five submissions before adjudication, mainly where participants flag uncertainty without choosing a source.
Interpretation. The task reveals distinct source-reconciliation approaches and supports further investigation of that behavior in these teams. It does not show that AI use caused the errors. It does not establish workforce prevalence, everyday transfer, or a training effect. The missing sessions remain outside the denominator.
Alternative explanations. The interface ranks the general rule first, task familiarity differs, volunteers may be more interested in AI than nonparticipants, and one rubric anchor does not clearly distinguish a justified hold from an incomplete answer.
Action. Repair the anchor and search presentation, then run an alternate-form practice test. Continue only if independent reviewers can apply the revised anchor consistently and critical missingness stays below the pre-specified threshold. The director owns the decision; the researcher owns the protocol and report; learning owns any module; governance reviews the evidence boundary.
Reporting QA
Two Prune can support a bounded study with a realistic assessment and evidence packet. It does not establish causal workforce outcomes, supply mature population norms, replace the research design, or make employment decisions.
- [ ] Stakeholder role and decision owner are explicit.
- [ ] Construct, population, task, tool, and period are frozen.
- [ ] Every indicator has a numerator, denominator, source, and missingness rule.
- [ ] Activity is not presented as capability.
- [ ] Groups and exclusions remain visible.
- [ ] Findings, interpretation, alternatives, and unknowns are separated.
- [ ] Recommendations name an owner and evidence gate.
- [ ] External statistics retain their definitions and authority.
- [ ] Versions and review dates are recorded.
- [ ] The report excludes employment recommendations.
Sources
OECD: AI measurement in ICT usage surveys
- OECD: AI surveys of employers and workers
- ILOSTAT: Statistical standards and methods
- U.S. GAO: Artificial Intelligence Accountability Framework
Sources: [1] [2] [3] [4]
Sources
- 1.OECD: AI measurement in ICT usage surveys · www.oecd.org
- 2.OECD: AI surveys of employers and workers · www.oecd.org
- 3.ILOSTAT: Statistical standards and methods · ilostat.ilo.org
- 4.U.S. GAO: AI Accountability Framework · U.S. Government Accountability Office