How to find missing edge cases in AI-enabled work

Draw the decision-boundary map

Start with the artifact or action the workflow produces. List the inputs, decision rules, outputs, and escalation conditions. Then mark every point where behavior changes: a threshold, missing field, conflicting source, user permission, time cutoff, exception, or system dependency.

The NIST AI RMF Core calls for testing under conditions similar to deployment settings and documenting test sets, metrics, and methods. A boundary map makes those conditions concrete before examples are chosen.

For an AI-assisted expense review, boundaries might include the approval threshold, foreign currency conversion date, missing receipt rule, duplicate transaction logic, restricted category, and manager override. “Ten normal expenses” cover volume, but not the decisions that distinguish correct from incorrect handling.

Sources: [1]

Generate cases from transitions, not labels

For each boundary, create five cases: just below, exactly at, just above, missing, and contradictory. Add a sixth case where the correct response is to stop and ask for help. This transition set often reveals gaps hidden by broad labels such as “complex request” or “unusual data.”

The NIST AI measurement and evaluation programme emphasizes challenge problems, meaningful datasets, and context-specific evaluation. Apply that logic at workflow scale. A case is useful when it tests a named behavior and has an independently checkable expected response.

Do not let the same example test every requirement. Separate data-quality cases from instruction conflicts, permission failures, and communication failures so a pass or failure can be interpreted.

Sources: [2]

Work a synthetic edge-case register

Suppose a participant must prepare an expense exception memo from a policy, spreadsheet, and manager email. The first packet contains only valid domestic expenses below the threshold. An edge-case register exposes what is missing:

Each row records the boundary, triggering evidence, expected behavior, unacceptable shortcut, reviewer, and result. The register is an artifact, not a score by itself.

• E-01: expense is one cent below the approval threshold. Expected response: normal route.
• E-02: expense equals the threshold. Expected response: apply the policy's exact inclusive rule.
• E-03: receipt is missing but the manager email says to proceed. Expected response: flag the conflict and request the named exception evidence.
• E-04: foreign currency date differs between receipt and ledger. Expected response: identify the governing conversion date before calculating.
• E-05: two transactions share amount and merchant but have different identifiers. Expected response: investigate rather than delete one automatically.
• E-06: policy version in the packet is superseded. Expected response: stop and obtain the current source.

Run a coverage and mutation review

Coverage review asks whether every material boundary has at least one useful test. Mutation review changes one fact at a time and checks whether the expected outcome changes appropriately. If changing the receipt status does not change the workflow's response, either the rule is irrelevant or the test has exposed a defect.

NIST's draft TEVV-Athlon Framework presents an adaptable approach to building evaluations from objectives, events, tools, and measurement concepts. Because it is an initial public draft, use it as a current research reference rather than a settled standard.

Record uncovered boundaries separately from failed cases. An uncovered case means the evaluation did not create an opportunity to observe the behavior. It should not be scored as if the participant or system attempted and failed it.

Sources: [3]

Decide when coverage is sufficient

Prioritize cases by consequence, likelihood, detectability, and novelty. Stop adding cases when remaining conditions are low consequence, already covered by an equivalent decision transition, or better tested by another control. Reopen the register when policy, data, model, permissions, or workflow steps change.

Use independent review for the most consequential expected outcomes. The reviewer should be able to trace the boundary to an authoritative source and explain why the expected response follows.

Two Prune can create realistic assessment packets with ambiguity, contradictions, and evidence traps, then retain the submitted artifact and available workflow evidence. It does not prove that every possible edge case was covered or that observed behavior will repeat in every workplace setting. Missing opportunities should appear as evidence limitations, not be treated as participant failure.