Verifying false precision in AI-enabled knowledge work

Identify what the number claims to measure

Write the quantity in plain language before checking decimals. Name the population, period, unit, inclusion rules, data source, estimation target, and decision the number informs. Distinguish a count from an estimate, a tested benchmark result from generalized performance, and a directional signal from a stable operational metric.

Inspect denominators, sample selection, missing data, category changes, repeated trials, and model or analyst assumptions. A value such as 83.47 percent may simply reflect spreadsheet formatting. Extra decimal places do not create evidence. A rounded figure can also be misleading if it hides a wide interval or a fragile assumption.

NIST's 2026 report Expanding the AI Evaluation Toolbox with Statistical Models explains why AI benchmark comparisons require a defined estimand and uncertainty reasoning. It notes that observed differences may reflect chance when uncertainty is not quantified. The report addresses evaluation methods, not every business metric.

Sources: [1]

Complete the precision challenge sheet

Record the draft number, quantity definition, population, period, source cells or records, formula, transformations, missingness, uncertainty method, interval or range, sensitivity variables, comparison basis, reporting digits, reviewer, and final wording. Add a field for evidence that would change the decision.

Use distinct dispositions: supported as shown, recompute, round, add range, narrow population, label directional, remove comparison, or insufficient evidence. Preserve full calculation precision internally while matching displayed precision to the evidence and decision. If an interval is reported, state what it represents and the assumptions behind it.

The NIST Information Quality Standards emphasize reproducibility, documentation, and quantitative statements of uncertainty for NIST measurement results. These principles are useful for disciplined reporting, but a business team must choose methods appropriate to its own data and context.

Sources: [2]

Worked sheet: a synthetic capability dashboard

A synthetic dashboard states that a cohort's AI-work quality improved from 71.36 to 74.92, a 5.0 percent improvement. The sheet reveals twelve participants, different tasks before and after training, two missing records, and a scoring revision between rounds. The displayed decimals exceed what the design can defend, while the percentage calculation creates an impression of comparability that the task changes undermine.

The reviewer removes the improvement claim. The dashboard reports the two task-specific distributions separately, labels the comparison exploratory, explains the missing records and scoring change, and lists the conditions required for a comparable follow-up. Individual evidence remains available for human interpretation without turning the small cohort into a mature benchmark.

A second number reports 2.347 hours saved from self-reported ranges rounded to half-hour increments. The sheet changes the display to a broad reported range and separates recall-based estimates from directly observed timing. This example is synthetic and demonstrates a review method, not a measured outcome.

Stress-test the final number

Recalculate from the original inputs and vary assumptions that could change the decision: missing-data treatment, outliers, filters, time period, denominator, scoring rule, repeated trials, or task mix. Ask whether another analyst can reproduce the result and whether the displayed digits remain stable under reasonable alternatives.

Review charts, labels, rankings, and narrative claims as well as cells. A narrow axis or ordered list can imply precision that the table avoids. Reopen the sheet when the dataset, task, model, population, scoring method, or decision threshold changes.

Two Prune can observe whether a participant challenges quantitative claims within a bounded assessment and can present the supporting evidence and limitations. It does not validate a metric, establish mature benchmarks, prove causal impact, or make employment decisions.