Practical guide
AI dashboard specification verification checklist
A dashboard specification is ready for implementation only when another person can trace every displayed number to a defined source, repeat each transformation, understand every visual claim, and identify who owns unresolved questions. A polished wireframe is not enough. This checklist tests the specification before build work begins and again before the dashboard is released. Use it with the.
A dashboard specification is ready for implementation only when another person can trace every displayed number to a defined source, repeat each transformation, understand every visual claim, and identify who owns unresolved questions. A polished wireframe is not enough. This checklist tests the specification before build work begins and again before the dashboard is released. Use it with the source inventory, calculation definitions, wireframes, access rules, and approval record. It is an operational quality aid, not proof that a dashboard is accurate in every future data state.
Freeze the test packet
Record these fields before reviewing the specification:
If the decision or source version changes during review, freeze a new packet. Mixing evidence from two versions makes a pass impossible to reproduce.
Digital.gov advises teams to know the audience, identify the message, use the right chart, provide context, and remove clutter. The U.S. Web Design System also treats data visualizations as interfaces that need clear titles, labels, accessible colour use, and alternatives for users who cannot perceive the visual alone. Those principles inform the claim and presentation checks below; they do not validate a particular organization’s data. Digital.gov: An introduction to data visualization U.S. Web Design System: Data visualizations
- [ ] dashboard name, decision supported, audience, and expected review cadence
- [ ] specification version, owner, review date, and intended release
- [ ] authoritative systems, tables, fields, refresh windows, and data owners
- [ ] metric catalogue and transformation logic
- [ ] filter, comparison, drill-down, export, and access behaviour
- [ ] named reviewer for data, calculations, design, accessibility, privacy, and release
Sources: [1] [2]
Run the SIGNAL checklist
SIGNAL is a pre-build verification sequence. Mark each item pass, fail, not applicable with a reason, or blocked with an owner.
### S — Source traceability
### I — Instruction and calculation integrity
### G — Graphic and interaction fit
### N — Narrative and claim boundary
### A — Access, approval, and auditability
### L — Limitations, contradictions, and lifecycle
- [ ] Every metric names the system, table or object, field, population, and source owner.
- [ ] The specification distinguishes a system of record from a convenience copy or analyst extract.
- [ ] Refresh timing, expected latency, timezone, and late-arriving data treatment are explicit.
- [ ] Join keys and deduplication rules are defined, including missing and many-to-many matches.
- [ ] Suppressed, deleted, corrected, or backfilled records have a stated treatment.
- [ ] Sample records can be followed from the source through the proposed display.
- [ ] Each displayed value has a formula, unit, grain, inclusion rule, and exclusion rule.
- [ ] Numerator, denominator, cohort boundary, and comparison period are unambiguous.
- [ ] Currency conversion, rounding, null handling, and divide-by-zero behaviour are specified.
- [ ] Totals, percentages, medians, percentiles, rolling windows, and targets can be recalculated independently.
- [ ] Filter combinations cannot silently change the meaning of a metric.
- [ ] Metric names do not imply causation, quality, or performance beyond the calculation.
- [ ] Each chart type matches the comparison: trend, ranking, distribution, composition, or relationship.
- [ ] Axes, baselines, scales, units, sort order, and truncation choices are visible and defensible.
- [ ] Titles state what is shown rather than announcing an unsupported conclusion.
- [ ] Tooltips, legends, annotations, and empty states add necessary interpretation.
- [ ] Keyboard operation, focus order, colour contrast, text alternatives, and tabular access are specified.
- [ ] Filters show their current state and have a predictable reset.
- [ ] Every annotation and summary sentence points to the metric or source that supports it.
- [ ] Descriptive evidence is not presented as proof of cause.
- [ ] Forecasts, model outputs, estimates, and observed values are visibly distinct.
- [ ] Small samples, incomplete periods, definition changes, and other material limitations are shown near the claim.
- [ ] The specification says what users may decide from the view and what requires further investigation.
- [ ] Reader-facing labels avoid internal jargon and ambiguous abbreviations.
- [ ] Audience roles map to view, filter, export, and administration permissions.
- [ ] Sensitive fields, small-group disclosure risks, download controls, and retention rules have owners.
- [ ] Metric changes require a versioned definition and review rather than a silent edit.
- [ ] Release approval identifies the data owner, product owner, and accountable decision maker.
- [ ] The implemented view will expose a data-as-of time, definition link, and feedback route.
- [ ] Review evidence can be retained without copying restricted source data into an unsuitable location.
- [ ] Conflicts between source systems or metric definitions are listed with a disposition.
- [ ] Known gaps have an owner, due date, user-visible treatment, and escalation threshold.
- [ ] Tests cover stale data, no data, unexpected categories, extreme values, and failed refreshes.
- [ ] A rollback or disable condition is defined for materially misleading output.
- [ ] Review cadence and triggers include source, policy, audience, or decision changes.
- [ ] Residual uncertainty is visible to approvers rather than hidden in implementation notes.
Copyable verification record
Use one row per metric, interaction, claim, or unresolved limitation.
| ID | Specification element | Source or rule | Test performed | Expected result | Observed result | Disposition | Owner / approver | Residual uncertainty | | --- | --- | --- | --- | --- | --- | --- | --- | --- | | DS-01 | metric, chart, filter, or claim | exact source and version | trace, recalculate, inspect, or challenge | measurable result | evidence found | pass / repair / hold / remove | named people | remaining question |
Attach the frozen source manifest, representative input records, hand calculation, wireframe, accessibility notes, contradiction log, and approval record. A screenshot can demonstrate a rendered state, but it cannot replace the metric definition or source trace.
Worked test: service-resolution dashboard
Consider a synthetic support dashboard specification with three tiles: “resolution rate,” “average handling time,” and “customer recovery.” The specification defines resolution rate as closed cases divided by opened cases for the selected month. It also permits filtering by region and channel.
The SIGNAL source test finds that reopened cases live in a separate event table and are not returned to the open population. The proposed tile would count a reopened case as resolved. The reviewer records a failed join rule, assigns the data owner, and requires a case-level trace through open, close, reopen, and final close events.
The calculation test then finds a cohort mismatch: the numerator uses cases closed in June while the denominator uses cases opened in June. A case opened in May and closed in June enters only the numerator. The team must choose and name a coherent measure—for example, the share of June-opened cases resolved within a defined period—and specify how immature cohorts appear. The checklist does not choose the business definition; it makes the choice and its consequence explicit.
The narrative test challenges “customer recovery.” The underlying field is a post-case satisfaction response available for only 18% of closed cases. The label implies more than the measure supports. The specification changes the tile to “post-case satisfaction among respondents,” shows response coverage, and holds regional comparison when the minimum reporting condition is not met.
Finally, the lifecycle test injects a delayed refresh. The wireframe has no stale-data state, so release remains blocked until the view shows its data-as-of time and suppresses the executive summary when the feed exceeds the defined latency threshold. The completed evidence packet contains two traced cases, a recalculation sheet, revised labels, the delayed-refresh state, approval owners, and the unresolved question about nonresponse.
Independent challenge before release
Give a reviewer who did not write the specification one important metric and one high-impact filter. Ask them to locate the source, reproduce the value from representative records, explain the visual claim, and predict the result of an edge case. Expand the challenge when they cannot do so without private guidance or when any source, formula, label, or access assumption is wrong.
Hold the specification when a controlling source is unavailable, the metric population cannot be stated, a material contradiction lacks an owner, sensitive access is unresolved, an important claim cannot be traced, or an edge case could create a materially misleading display. A checklist pass means the specification is inspectable and ready for its named build decision; it does not certify future data quality or the suitability of every decision made from the dashboard.
Two Prune can place dashboard-specification work inside a realistic AI-enabled assessment and evaluate how a participant frames the decision, navigates data, uses AI, verifies calculations and claims, manages risk, and communicates a deliverable. It does not operate production dashboards or make employment decisions.
Sources
Digital.gov: An introduction to data visualization
- U.S. Web Design System: Data visualizations
Sources: [1] [2]
Sources
- 1.Digital.gov: An introduction to data visualization · U.S. General Services Administration
- 2.U.S. Web Design System: Data visualizations · designsystem.digital.gov