How to design an operational dashboard with AI

Inventory decisions, not data

Interview the people who run the process. For each recurring decision record who makes it, when, what question they answer, what evidence they need, and what action follows. Examples include whether to add capacity today, investigate a queue now, or revise a monthly process.

The GOV.UK Service Manual recommends basing performance metrics on a service’s purpose, developing hypotheses, identifying data sources, and giving measurements context. That sequence is useful because it prevents available data from defining the dashboard’s purpose.

Write metric contracts

Every metric needs a definition that another analyst can reproduce:

If two teams use “completion rate” with different denominators, display them as different metrics until the definitions are reconciled.

• name and decision served;
• owner and review cadence;
• numerator, denominator, unit, and aggregation;
• inclusion and exclusion rules;
• source system and transformation;
• timezone, freshness, and late-data treatment;
• threshold and response;
• known limitations.

Worked synthetic specification

Imagine an assessment-operations team needs to decide whether a live support queue requires intervention.

| Metric | Decision | Definition | Trigger | Owner | | --- | --- | --- | --- | --- | | Oldest unresolved critical case | Escalate support | Age since verified critical-case creation, excluding resolved cases | Over 30 minutes | Incident lead | | Submission processing delay | Inspect pipeline | p95 time from recorded submission to report-ready state | Over 10 minutes for 15 minutes | Platform operator | | Evidence-processing failure rate | Pause affected workflow | Failed evidence jobs / eligible jobs, 15-minute window | Over 2% and at least 5 failures | Engineering on-call |

These thresholds are synthetic operating choices, not performance recommendations.

The dashboard should link a threshold breach to a runbook: confirm data freshness, inspect the affected segment, assign an incident owner, and record the decision. A metric without an action is likely informational rather than operational.

Design layers by cadence

Use a live layer for immediate state and alerts, a daily layer for capacity and backlog, and a trend layer for process improvement. Do not cram all three into one screen. Show definitions and last-updated times near the measures. Pair rates with meaningful counts so small denominators are visible.

AI can review the draft for duplicate metrics, thresholds without owners, measures unsupported by source fields, and opportunities to segment a failure. It can also generate test cases from the contracts. Owners must validate business meaning.

Test the dashboard before launch

Seed known scenarios: stale source data, zero denominator, duplicate events, late arrivals, threshold crossings, and recovery. Confirm the dashboard displays uncertainty or staleness rather than a reassuring value. Compare selected numbers with source records and an independent query.

Run a tabletop: “The failure rate is red but the source is 40 minutes stale. What decision should the operator make?” If the answer is unclear, the specification needs a freshness rule and escalation path.

Verification checklist

Ensure every measure serves a named decision; definitions are reproducible; owners and cadence are visible; thresholds produce actions; freshness and data quality are explicit; rate denominators are inspectable; access controls match sensitivity; and test fixtures cover failure states.

Sources and claim map

The dashboard, metrics, and thresholds are synthetic. This educational specification does not claim validated operational performance or assured incident detection.

• GOV.UK Service Manual, How to set performance metrics for your service: supports starting from purpose, benefits, hypotheses, sources, context, and iteration.
• GOV.UK Service Manual, Using performance data to improve your service: supports connecting measurements to improvement decisions and segment investigation.

Sources: [1] [2]