How to develop and assess calculation checking in AI-enabled work

Separate arithmetic correctness from decision fitness

A calculation can be arithmetically correct and still mislead. It may use the wrong population, time period, exchange rate, denominator, sign convention, version, or business rule. It may also round too early, double-count a category, omit a boundary case, or combine quantities with incompatible units.

The 2025 UK AQuA Book distinguishes verification from validation. Verification checks whether analysis satisfies its specified design requirements; validation asks whether the analysis meets the needs of intended users and the intended environment. The same distinction is useful for a spreadsheet, model output, or AI-generated calculation:

Begin with the question the number is meant to answer. Then record the source values, units, date or period, inclusion rules, formula, transformations, rounding rule, and expected output. If those elements are unavailable, the number is not ready for a meaningful check.

An AI assistant can explain a formula, generate test cases, or independently calculate a small example. It must not be treated as an independent check when it is repeating the same unsupported inputs or the same hidden assumptions. Independence comes from a separately constructed path, authoritative records, or a reviewer who can inspect the logic.

• verification: Did the implemented formula and data handling produce the specified result?
• validation: Is that specified result the right measure for the question and decision?

Build a RECAP check sheet

Use RECAP to retain the numerical audit trail:

| Field | Question | Evidence to keep | | --- | --- | --- | | Requirement | What exactly should the number represent? | Decision question, definition, scope, and tolerance | | Entries | Which input values, units, dates, and sources feed it? | Source references and extracted values | | Calculation | Which formulas, joins, filters, conversions, and rounding rules apply? | Human-readable expression and implementation location | | Alternate path | Can the result be reproduced independently or reconciled to a control total? | Separate calculation, subtotal, or authoritative comparison | | Probes | Which boundary, sign, missing-value, duplication, and magnitude tests were run? | Test cases, expected results, actual results, and disposition |

Write the human-readable expression before inspecting code or a generated formula. For a weighted average, state the numerator, denominator, population, and treatment of missing values. For a percentage change, state the baseline and what happens when it is zero or negative. For a currency total, state the conversion date, rate source, and whether rounding occurs per item or after aggregation.

The alternate path should differ materially from the original. Re-entering the same formula in a second cell is weak independence. Better checks include summing verified subtotals, recalculating a sample from raw records, using a simple hand-worked case, or reconciling the result to a known control total.

Probes target predictable failure modes. Use a zero, negative, very large, missing, duplicated, and just-inside or just-outside threshold case where relevant. Add a dimensional check: dollars divided by people produces dollars per person, not a percentage. Add a magnitude estimate before exact calculation. If the precise answer is 100 times larger than the rough order of magnitude, investigate before explaining it away.

Worked artifact: checking a service-cost comparison

Consider an original synthetic example. An operations team compares the monthly cost of processing service requests under a current workflow and a proposed assisted workflow. An AI assistant produces this result:

> Current cost per resolved request: $18.40. Proposed cost: $15.10. Estimated saving: 17.9%.

The source pack contains monthly labor hours, hourly cost, software cost, requests opened, requests resolved, and reopened requests. The RECAP sheet begins with the requirement: compare total monthly operating cost per finally resolved request for the same reporting period. “Finally resolved” means a request that is closed at month end and not reopened within the measurement window.

The entries check finds:

The assistant had divided current cost by all closed requests but divided proposed cost by closed requests minus reopened requests. It also omitted software cost from the proposed workflow. The arithmetic inside each expression was valid, but the comparison used different denominators and incomplete inputs.

The check sheet states one consistent expression:

> Cost per finally resolved request = (labor hours × hourly cost + monthly software cost) ÷ (closed requests − reopened requests)

The independent reconstruction gives:

These are synthetic values and not a customer result or product benchmark. The corrected calculation does not establish that the proposed workflow is preferable. It excludes transition effort, quality, waiting time, exception handling, and uncertainty about reopened requests outside the window.

The alternate path reconciles both numerators to the labor ledger and software invoice total, then checks ten raw request records against the closed and reopened counts. The probes include a month with zero finally resolved requests, a negative correction in the labor ledger, a duplicate request identifier, and a request reopened just after the window. The zero case must return “not computable,” not infinity or zero cost. The duplicate must be excluded or resolved before the denominator is accepted.

The final artifact records a disposition: the original saving claim is withdrawn; the corrected cost comparison is retained with its definition and limits; and the decision owner receives a separate request for quality and transition evidence.

• current workflow: 420 labor hours at $42 per hour, $900 software cost, 1,050 requests closed, and 70 reopened;
• proposed workflow: 360 labor hours at $42 per hour, $2,400 software cost, 1,080 requests closed, and 120 reopened.
• current: (420 × 42 + 900) ÷ (1,050 − 70) = 18,540 ÷ 980 = $18.92 after final rounding;
• proposed: (360 × 42 + 2,400) ÷ (1,080 − 120) = 17,520 ÷ 960 = $18.25;
• relative change: (18.25 − 18.92) ÷ 18.92 = -3.5% after rounding.

Practice with a calculation mutation drill

Give a learner a small, correct calculation and its source records. Ask for a RECAP sheet. Then introduce one mutation without identifying it:

The learner should detect the discrepancy, identify which field of the check sheet failed, quantify the effect, and revise every downstream statement that depends on the result. Reward a clear stop when the available records cannot resolve the issue.

AI can generate adversarial test cases or translate a formula into plain language. The participant should verify that each test is relevant and should perform at least one independent reconstruction without relying on the original generated expression. Retain the source values, the original result, the test records, and the corrected output.

• swap a monthly input for an annual figure;
• change a percentage from 5% to 5 percentage points;
• duplicate a row during a join;
• move rounding from the final total to each line item;
• replace resolved cases with opened cases in the denominator;
• omit a negative value from a filtered range.

Assess calculation checking from observable work

Review the check sheet, source trace, tests, corrections, and final communication:

Do not assess only whether the final number matches an answer key. A participant may catch a flawed specification that the expected result reproduces. Conversely, arriving at the expected number by copying the same formula is weak evidence of verification.

• requirement fit: the participant defines what the number should represent and why it is used;
• input control: values retain provenance, units, periods, versions, and inclusion rules;
• formula transparency: transformations can be inspected in plain language and in the implementation;
• independent reconstruction: at least one materially separate path reproduces or challenges the result;
• boundary coverage: tests exercise missing, zero, negative, duplicate, extreme, and threshold cases where relevant;
• dimensional reasoning: units and denominators remain coherent through every step;
• error propagation: dependent claims and tables change when the calculation changes;
• limit communication: unresolved uncertainty and excluded consequences remain visible.

Verify the check before closing it

First, rerun the calculation from retained inputs in a clean environment or simple independent worksheet. Compare exact values before rounding. Reconcile totals to authoritative controls and inspect any unmatched remainder. Confirm that filters and joins preserve the intended population and that repeated runs use the same versioned data.

Second, perform a reasonableness review. Estimate the answer's sign and order of magnitude, compare it with a relevant baseline, and explain large deviations using evidence. Third, validate the use: confirm the metric actually answers the decision question and does not hide a consequential quality, timing, or distribution issue.

The AQuA Book says analytical assurance should be proportionate to the complexity and risks of intended use, and it includes analyst testing, peer review, and reconciliation to independent sources among assurance activities. A low-consequence internal estimate may need a simple reconstruction and probe set. A high-consequence or irreversible decision needs stronger independence, documented review, and an escalation path.

Calculation checking can reveal errors and limitations in a bounded result; it cannot establish that every relevant input or consequence has been captured. In a Two Prune assessment, the RECAP sheet and revisions can provide task-bounded evidence about Verification & Risk Control, Evidence Navigation, and Judgment & Synthesis. They do not establish general workplace performance and do not make an employment decision.

Sources

UK Government: The AQuA Book

• UK Government: Verification and Validation for the AQuA Book
• UK Government: AQuA Book resources

Sources: [1] [2] [3]