How to develop and assess exception handling in AI-enabled work

Define the normal path before looking for exceptions

An exception exists relative to an expected process. Start by describing the normal input, permitted transformation, expected output, responsible role, and acceptance check. Without that baseline, teams may treat any surprising result as a crisis or, worse, normalize a material failure because it occurs often.

The NIST AI Risk Management Framework Playbook recommends tracking reported errors, complaints, response times, response types, overrides, policy exceptions, escalations, and accountable go or no-go decisions. UK government AI assurance guidance likewise calls for clear reporting lines, avenues for escalation, empowered staff, and processes to identify, manage, and mitigate risks. These sources support a practical lesson: the exception process should be designed and evidenced before the difficult case arrives.

Separate three concepts:

The labels are context-dependent. A missing optional formatting field may be routine variation in a draft. A missing source for a consequential claim may be an exception requiring return to the author. Exposure of restricted information is not merely a content exception and should follow the organization's security or privacy incident procedure.

• variation remains within the stated operating range and can continue through the normal check;
• exception breaches a condition or presents ambiguity that requires a defined alternate path;
• incident produces or threatens material harm and activates the relevant incident process.

Use a SAFER exception record

SAFER is a six-part record that connects detection to accountable resolution:

| Field | Question | Evidence to retain | | --- | --- | --- | | Signal | What observable condition departed from the normal path? | Output excerpt, check result, or event reference | | Affected boundary | Which task, person, dataset, system, or decision may be affected? | Bounded scope and known exclusions | | First control | What immediate action prevents further propagation? | Pause, quarantine, rollback, or manual hold | | Escalation | Who has the authority and expertise to decide the next step? | Named role, route, urgency, and acknowledgement | | Resolution | What disposition was chosen and on what evidence? | Correct, accept, rerun, bypass, retire, or investigate | | Re-entry | What must be true before normal work resumes? | Verification check, owner approval, and follow-up date |

The first control should be reversible where practical. It is usually safer to pause one affected record than to disable an entire workflow without understanding scope. The record should not ask a frontline worker to make a legal, security, or technical determination outside their authority. It should help them recognize the signal, contain it within their allowed actions, and hand it to the role that owns the decision.

Severity and urgency are different. A potentially severe problem may be contained and able to wait for a specialist. A lower-severity issue may be urgent because it blocks a time-sensitive deliverable. Record both rather than collapsing them into one red, amber, or green label.

Worked artifact: a conflicting policy summary

Consider a synthetic operations team that uses an AI assistant to summarize policy documents into a response checklist. This example is original and does not describe a Two Prune customer, measured outcome, or production incident.

The normal path requires the worker to identify the current policy version, generate a draft checklist, compare every mandatory item with the source, and record the source section. During review, the worker sees that the draft says approvals are valid for 24 months, while the current policy states 12 months. An archived version used 24 months.

The worker opens a SAFER record:

The worker does not simply change 24 to 12 and continue. That would repair the visible output but leave the shared cause and affected scope unknown. Nor does the worker declare the whole assistant unsafe. The record narrows the problem, preserves the evidence, and gives each decision to the appropriate owner.

The team adds a follow-up action: whenever a summary cites a superseded policy, the workflow should route it for review rather than silently prefer the newest document. This is a design improvement, not proof that every future version conflict has been eliminated.

• Signal: the validity period conflicts with section 4.2 of the current policy.
• Affected boundary: the current checklist draft and three drafts generated from the same archived attachment may be affected. No delivered checklist has yet been identified.
• First control: mark the four drafts as held and prevent them from being copied into the approved template.
• Escalation: route the case to the policy owner and workflow maintainer, with the current and archived passages attached. The policy owner decides the correct rule; the maintainer owns retrieval configuration.
• Resolution: the policy owner confirms 12 months. The four drafts are corrected. The maintainer removes the archive from the default retrieval set but preserves it in the records system.
• Re-entry: rerun a seeded check that asks about the validity period, verify the answer against the current source, and obtain the workflow maintainer's acknowledgement before releasing the held drafts.

Practice with exception drills

Use short drills in which the correct response changes with the evidence. Examples include a unit mismatch in a spreadsheet, a source that cannot be opened, personally identifying information pasted into the wrong workspace, an output that exceeds an approved scope, and a request that conflicts with policy.

For each drill, ask the learner to produce a SAFER record and narrate the first five minutes of action. Change one condition after the initial response: reveal that the output was already sent, show that the source conflict affects only one region, or establish that the worker lacks authority to quarantine a shared system. The learner should revise the boundary, control, and route instead of defending the first answer.

AI can help enumerate plausible affected records, draft a handoff, or propose verification tests. The worker should verify the facts and choose the disposition. An AI-generated escalation message is not evidence that the right owner received or accepted the case.

Assess observable exception work

Use the participant's actions and retained artifact rather than rewarding dramatic language:

Retain the initial signal, SAFER record, communications, disposition, re-entry result, and any workflow change. A closed ticket alone is weak evidence because it may not show whether the case was contained, whether the owner made a decision, or whether the same error remained active elsewhere.

• signal recognition: the departure is described with observable evidence;
• classification: variation, exception, and incident routes are not confused;
• scope control: affected and unaffected boundaries are stated and revised as evidence changes;
• containment: the first action limits propagation without unnecessary disruption;
• authority matching: the handoff reaches a role able to decide the issue;
• evidence preservation: source versions, output state, checks, and timestamps remain inspectable;
• resolution discipline: correction, acceptance, bypass, investigation, or retirement has a rationale;
• safe re-entry: normal processing resumes only after a named check and acknowledgement.

Verify the exception path

Trace the case in both directions. From the signal, confirm that every affected item reached a disposition. From each disposition, confirm the evidence and authority that support it. Test the re-entry condition independently of the corrected example where possible. Then ask whether the exception revealed a missing normal-path control, an unclear boundary, or only a genuinely rare condition.

Review the route itself. Did the named owner acknowledge the handoff? Could staff raise the concern without bypassing access controls? Would the evidence survive if a chat thread disappeared? Did the resolution introduce a second risk? These questions test the operating system around the decision, not just the prose in the record.

Exception handling reduces the chance that an abnormal case passes silently through a workflow. It does not promise that all failures will be detected or resolved without specialist judgment. In a Two Prune assessment, the exercise can provide task-bounded evidence about Verification & Risk Control, Judgment & Synthesis, and Communication & Delivery. It remains an assessment of observed work in the simulation, not a safety certification or employment decision.

Sources

NIST AI Risk Management Framework Playbook

• UK Government: Introduction to AI assurance
• NIST AI Risk Management Framework 1.0

Sources: [1] [2] [3]