How to develop and assess contradiction detection in AI-enabled work

Preserve conflict before trying to resolve it

The 2024 US Government Auditing Standards require an overall assessment of collective evidence and call for additional procedures when significant limitations or uncertainties appear. They specifically point to corroborating evidence and the relationship between evidence, objectives, findings, and conclusions. NIST's AI Risk Management Framework likewise emphasizes documented context, limitations, uncertainty, testing, and monitoring rather than treating an AI output as self-validating.

A contradiction is not simply two sentences that use different words. Claims may differ because they describe different dates, populations, measures, versions, or levels of authority. Conversely, two documents can use similar language while relying on incompatible definitions. Detection therefore requires preserving each claim with its context before asking which one should drive the work.

AI can help locate candidate conflicts across a large pack, but it may also merge them, choose the more fluent source, or invent a reconciliation. The worker remains responsible for source retrieval, comparison, and the final treatment.

Build a CLASH register

Record each material conflict in a CLASH register:

| Field | Diagnostic question | Evidence artifact | | --- | --- | --- | | Claims | What exactly does each source assert? | Short quotation or faithful paraphrase with locator | | Lens | Do the claims use the same definition, unit, population, and time period? | Normalized comparison fields | | Authority | Who produced each source, for what purpose, and with what approval status? | Provenance and authority note | | Sequence | Could version, effective date, or later evidence explain the difference? | Timeline and supersession record | | Handling | Is the conflict resolved, bounded, escalated, or left open? | Disposition, rationale, and affected output |

Never begin with a merged summary. Capture the claims separately so that a reviewer can inspect the original difference. Distinguish a direct contradiction from a tension, omission, or change over time. A newer source does not automatically prevail if it addresses a different question, lacks approval, or cites weaker evidence.

The disposition should state what the conflict changes. It may block a conclusion, narrow a calculation, require two scenarios, or affect only background wording. “Needs review” is incomplete unless the owner and decision point are named.

Worked example: a synthetic renewal evidence pack

Consider a synthetic analyst preparing a supplier-renewal note. The service dashboard reports 98.7 percent availability, an incident log records a six-hour outage, the contract summary says the target is 99.5 percent, and an email says the supplier “met service levels.” This example is original and does not describe a Two Prune customer.

An AI summary states that performance was broadly compliant except for one outage. The analyst does not accept that reconciliation and opens a CLASH register:

The analyst rebuilds a small reconciliation table showing event duration, classification, included minutes, denominator, and source. One incident is planned maintenance under the contract; another has no approved classification. The note therefore presents a provisional range and identifies the evidence needed for a final determination. It does not repeat the email's conclusion or claim breach.

The artifact preserves why the numbers differ and what decision remains open. The contradiction changes the certainty and next action, not merely the wording of the summary.

• Claims: the dashboard shows 98.7 percent for the quarter; the email says service levels were met; the contract summary lists 99.5 percent; the incident log records six hours;
• Lens: the dashboard excludes planned maintenance, the incident log includes all lost access, and the email does not name a period or metric;
• Authority: the signed contract extract governs the target, the dashboard is an operational calculation, the log is event evidence, and the email is an unsubstantiated stakeholder statement;
• Sequence: all records concern the same quarter, but the dashboard was regenerated after one incident classification changed;
• Handling: compliance remains unresolved until the calculation definition and approved exclusions are checked against the signed contract and source events.

Practice with controlled conflicts

Give a learner four sources containing different conflict types:

Ask for a CLASH register before a conclusion. The learner should resolve the first three through normalization and provenance while keeping the fourth visible. Then add an AI-generated synthesis that confidently chooses one source without explanation. Require the learner to diagnose what context the synthesis lost.

Practice should include both false positives and subtle conflicts. If every difference is escalated, the process creates noise. If only direct numerical mismatches are recorded, material definition and authority conflicts will be missed.

• the same measure for different periods;
• the same label with different definitions;
• a later approved version that supersedes an earlier one;
• two equally relevant sources that genuinely disagree.

Assess observable contradiction-detection behavior

In an assessment, include several evidence types and at least one contradiction that could materially change the deliverable. Observe whether the participant:

Do not award credit merely for finding many differences. The relevant behavior is identifying conflicts that matter to the task and handling them proportionately. A participant may notice a contradiction but still choose an unsupported resolution; detection and resolution should be reviewed separately.

• captures incompatible claims with precise source locations;
• checks definition, unit, population, period, and version before declaring conflict;
• distinguishes authority and purpose from confidence or writing quality;
• seeks corroborating evidence or reconstructs the underlying calculation;
• avoids allowing AI to blend unresolved claims into one account;
• records whether the conflict is resolved, bounded, escalated, or open;
• changes the conclusion, confidence, or next action when the evidence requires it;
• communicates unresolved conflict without overstating either source.

Verify through adversarial reconciliation

Ask an independent reviewer to take the opposite provisional position and identify which evidence would support it. If the reviewer can expose an omitted definition, version, or source, reopen the CLASH entry. Recalculate any normalized values from the original records rather than from the AI summary.

Then inspect the final artifact for conflict leakage. Every material conclusion should either use a resolved evidence path or disclose the remaining limitation. Search for categorical statements where the register contains an open conflict. Confirm that superseded sources remain traceable but do not silently drive current claims.

Contradiction detection is task-dependent and does not establish that one source is true in every context. In a Two Prune assessment, the register, source review, AI interactions, recalculation, revisions, and final artifact can contribute bounded evidence about Evidence Navigation, Verification & Risk Control, Judgment & Synthesis, and Communication & Delivery. They do not make an employment decision.

Sources

US GAO Government Auditing Standards 2024 Revision

• NIST AI Risk Management Framework Core

Sources: [1] [2]