Practical guide

AI capability baseline template: free template and guide

An AI capability baseline is a dated description of what people can demonstrate on representative work before a programme, policy, or workflow change. It is not a confidence survey, a tool-usage count, or a permanent label. A useful baseline preserves the task conditions, evidence, scoring rules, missingness, and uncertainty so a later comparison is interpretable. This copyable template uses.

By Two Prune

An AI capability baseline is a dated description of what people can demonstrate on representative work before a programme, policy, or workflow change. It is not a confidence survey, a tool-usage count, or a permanent label. A useful baseline preserves the task conditions, evidence, scoring rules, missingness, and uncertainty so a later comparison is interpretable. This copyable template uses BASE: boundary, activities, signals, and evidence statement.

Source basis and claims boundary

NIST's AI Risk Management Framework describes measurement as a combination of quantitative, qualitative, or mixed methods grounded in the context of use, with documented uncertainty, benchmarks, reporting, and independent review. NIST's 2026 TEVV-Athlon framework organizes evaluations around measurement concepts, events, tools, and data rather than one undifferentiated score. The UK Government's 2025 AQuA Book supports fit-for-purpose analysis, proportionate assurance, traceable assumptions, distinct analyst and assurer roles, and explicit limits. These sources inform the controls below. They do not validate this template, define a universal AI-capability construct, or make one baseline comparable across different tasks and populations. NIST AI RMF NIST TEVV-Athlon AQuA Book

Sources: [1] [2] [3]

Copyable BASE record

### Record control

| Field | Entry | | --- | --- | | Baseline title and version | | | Owner, reviewer, and approver | | | Population and sampling rule | | | Assessment window | | | Intended comparison or decision | | | Prohibited interpretations | | | Data classification and retention | | | Next review or expiry date | |

### B - Boundary

Define the exact capability question and the conditions under which evidence will be collected.

This baseline describes observed performance by [population] on [representative tasks] with [tools, sources, time, and support]. It is intended to inform [bounded use]. It does not establish [non-claim].

| Included capability component | Excluded component | Why the distinction matters | | --- | --- | --- | | | | |

Record prerequisites, accessibility accommodations, AI permissions, source availability, time limits, prior training, and any material differences between participants. A baseline loses meaning if the later comparison changes several of these conditions without recording the change.

### A - Activities

Choose a small set of authentic activities that create opportunities to show the target behaviours.

| Activity | Work product | Inputs and tools | Judgment point | Verification opportunity | Coverage | | --- | --- | --- | --- | --- | --- | | | | | | | |

Use more than one activity when one task cannot represent the construct. Avoid tasks that mainly reward topic familiarity, writing speed, or prior access to a particular tool. Run a walkthrough to confirm that the source set is complete, instructions are understandable, and the intended behaviours are actually possible.

### S - Signals and scoring

Map each claim to observable evidence and a rule for uncertainty.

| Signal | Direct evidence | Anchor | Missingness rule | Rival explanation | Confidence note | | --- | --- | --- | --- | --- | --- | | | | | | | |

Useful signals may include framing the decision, selecting relevant sources, setting appropriate AI boundaries, checking calculations and citations, revising after contrary evidence, explaining uncertainty, and producing a usable deliverable. Tool frequency, prompt length, self-reported confidence, and polished prose are not substitutes for those behaviours.

Define scoring anchors before reviewing results. Require evidence citations for judgments. Specify double-scoring or adjudication for uncertain cases, and keep missing evidence separate from evidence of weak performance.

### E - Evidence statement

Summarize the result without erasing task conditions or uncertainty.

| Component | Observed pattern | Evidence strength | Important limitation | Follow-up evidence | | --- | --- | --- | --- | --- | | | | | | |

Include sample size, completion and missingness, task versions, score distribution only where interpretable, notable failure patterns, scorer disagreement, and the comparison that would be legitimate later. Keep individual evidence protected and report aggregates only when group size and data governance allow it.

Worked example: fictional policy-update baseline

A service-operations team is preparing an AI-support programme. Twelve participants complete two 35-minute activities: produce a policy-change briefing from four internal documents, then review an AI-drafted customer response containing an outdated rule and an unsupported citation. Everyone receives the same source set, desktop tools, AI permissions, and instructions.

The boundary states that the baseline describes performance on these two simulated tasks before training. It is not an estimate of productivity, general intelligence, job performance, or future learning. One participant uses an approved accessibility accommodation; the record notes the condition without treating it as lower capability.

The signal map covers decision framing, source selection, AI orchestration, verification, synthesis, and communication. A verification anchor requires opening the cited source, detecting the superseded rule, rejecting or repairing the fabricated citation, and documenting the correction. Merely saying “I checked it” does not meet the anchor.

The first review finds that three participants never opened a missing appendix because it was absent from their source bundle. The team marks those observations as administration missingness, repairs the bundle, and reruns that activity. It does not score the omission as poor evidence navigation.

The evidence statement reports that source verification was the clearest shared development area in this bounded sample. It records scorer disagreement on two submissions, the adjudication rule, and the exact task version. A later programme comparison must use equivalent activities and conditions or explain why the change prevents a direct comparison.

Practitioner QA

Is the intended use narrower than the evidence actually collected?

  • Do the activities represent meaningful work rather than generic AI trivia?
  • Can every score claim be reconstructed from observable evidence?
  • Are self-report, usage, workflow, and work-quality signals kept distinct?
  • Are missingness and administration failures separated from weak performance?
  • Are scorer calibration, disagreement, and confidence recorded?
  • Would a later comparison preserve or document changes in task conditions?
  • Are privacy, accessibility, retention, and access controls defined?

Two Prune boundary

Two Prune provides realistic AI-enabled work simulations and structured assessment evidence for human interpretation. It can support a task-bounded baseline when the design and evidence are appropriate. It does not validate the baseline, prove programme impact, establish general capability, make employment decisions, or turn one cohort's results into a mature benchmark.

Sources

NIST: AI RMF Core

  • NIST: TEVV-Athlon Framework for Evaluating AI Systems
  • GOV.UK: The AQuA Book

Sources: [1] [2] [3]

Sources

  1. 1.NIST AI RMF · National Institute of Standards and Technology
  2. 2.NIST TEVV-Athlon · National Institute of Standards and Technology
  3. 3.AQuA Book · UK Government