Practical guide

What evidence shows output labeling is operating?

Evidence that output labeling is operating must show that the right outputs receive the intended disclosure, that the disclosure stays associated with the output across relevant transformations, that users can perceive and interpret it in context, and that exceptions trigger repair. A policy, interface screenshot, or successful demonstration proves design or configuration only; it does not.

By Two Prune

Evidence that output labeling is operating must show that the right outputs receive the intended disclosure, that the disclosure stays associated with the output across relevant transformations, that users can perceive and interpret it in context, and that exceptions trigger repair. A policy, interface screenshot, or successful demonstration proves design or configuration only; it does not establish sustained operation.

Source and control boundary

NIST AI 100-4 reviews technical approaches to digital content transparency, including provenance tracking, watermarking, detection, labels, and user-interface disclosures, while noting modality, context, implementation, and user-experience limits. The NIST AI RMF describes transparency as role- and lifecycle-sensitive and cautions that transparency does not itself make a system accurate, secure, privacy-enhanced, or fair. These sources support test design, not legal compliance or universal labeling requirements. NIST: Reducing Risks Posed by Synthetic Content NIST AI RMF: Accountable and Transparent

Sources: [1] [2]

Test operation with LABEL

### L — Label rule, scope, and lineage

Define which output types, channels, users, uses, and transformations require a label. Record exact label text or symbol, placement, timing, language, metadata, provenance record, and responsible owner.

Map the output lifecycle: generation, editing, export, copy, screenshot, conversion, embedding, publication, download, syndication, archive, and deletion. Identify where association can be lost.

An excluded channel needs a named alternative control. A statement that “all AI output is labeled” is not testable without a population and rule.

### A — Applicable population and coverage test

Construct a population from system events, output records, publication records, or another authoritative source independent of the label itself. Sample across output modality, route, model or tool, template, language, device, user type, and transformation.

For each sampled output, test whether a label was required, applied, correct, visible, timely, and retained. Reconcile counts:

| Measure | Definition | | --- | --- | | eligible outputs | outputs meeting the frozen rule | | correctly labeled | eligible outputs with correct content and placement | | false negatives | eligible outputs missing or losing the label | | false positives | outputs labeled when the rule does not apply | | indeterminate | insufficient evidence to decide |

Do not calculate a coverage rate by dropping indeterminate cases from the denominator without explanation.

### B — Binding through transformations

Test whether the disclosure remains connected to the relevant content when users edit, crop, convert, copy, export, quote, print, or move it to another channel. A visible banner may not travel with downloaded content; metadata may be stripped; watermarks may be damaged; a copied paragraph may lose page-level context.

Record the expected binding for each transformation and the fallback when persistence is not technically possible. Test actual paths, not just the generation screen.

### E — Experience, comprehension, and burden

Test whether intended users can notice, access, and interpret the disclosure under realistic conditions. Include keyboard and assistive-technology access, zoom, contrast, small screens where the public surface supports them, language, cognitive load, and time pressure.

Ask task-specific comprehension questions rather than “Did you see the label?” A user should understand what involvement is disclosed, what is not disclosed, and what action the label suggests. A label that users interpret as a quality or truth certification can create a new risk.

Measure operational burden: false alerts, repeated acknowledgements, support requests, publishing delay, manual repair, and workarounds. Burden does not automatically justify removal; it informs redesign and proportionality.

### L — Loss, exceptions, and lifecycle monitoring

Define exception states: missing label, wrong label, stale text, inaccessible disclosure, broken provenance link, stripped metadata, unsupported channel, duplicate label, or user override. Each exception needs severity, owner, correction, affected population, due date, and closure test.

Monitor configuration changes, new output types, channel changes, model or tool changes, template updates, accessibility defects, incidents, user complaints, and measured label loss. Trigger re-test when the rule or lifecycle changes.

Retain the control objective, rule version, population query, sample, test scripts, output evidence, transformation results, user study, exceptions, decisions, and review date.

Evidence ladder

Higher rungs do not erase failures below them. A user study cannot repair missing labels; perfect configuration cannot prove output coverage.

  • Policy evidence: the rule and authority exist.
  • Design evidence: label content and placement are specified.
  • Configuration evidence: the tested system is set to apply it.
  • Transaction evidence: sampled eligible outputs received it.
  • Persistence evidence: association survived relevant transformations.
  • User evidence: intended users perceived and interpreted it.
  • Monitoring evidence: losses and changes are detected and repaired.

Worked control test: synthetic campaign-image workflow

A synthetic communications team uses an approved image generator for campaign concepts. Policy requires a visible “AI-assisted concept” label on internal review exports and a provenance record for any image moving to production design.

The owner reports 100 percent compliance based on a screenshot of the generation interface. LABEL builds an independent population of 240 exported concepts over four weeks. The rule applies to 196; 44 are human-only comparison images.

A stratified sample covers PNG and JPEG export, direct download, presentation insertion, cropping, color correction, and production handoff. Forty-seven of 50 eligible sampled concepts show the visible label at first export. Two older templates omit it and one batch export places it outside the crop boundary.

Transformation testing finds that presentation insertion retains the visible mark, but a standard crop removes it from six of ten tests. JPEG conversion strips the optional provenance metadata in the current toolchain. The team therefore cannot infer downstream persistence from first-export coverage.

An accessibility review finds that the light-gray label fails contrast against two common backgrounds. Screen-reader users receive no equivalent description when the image is inserted without alt text. A comprehension exercise shows that three of twelve reviewers read “AI-assisted” as meaning the concept was approved for external use.

The control disposition is operating with material exceptions; repair before external production handoff. Actions move the label into a protected border, add contrast-safe styling and required alt-text guidance, block older templates, preserve provenance in the asset record rather than relying only on file metadata, and add a handoff confirmation.

The evidence packet contains the rule and version, independent population query, sample rationale, 50 output checks, transformation matrix, accessibility results, comprehension notes, seven exception records, owners, closure tests, and a 30-day retest date.

The test does not claim the content is authentic, accurate, safe, or legally sufficient. It shows where the organization’s stated disclosure control did and did not operate for the sampled workflow.

Acceptance and hold conditions

Accept an operating conclusion only when scope and population are defined, sampled evidence is reproducible, relevant transformations are tested, users can access and interpret the label, exceptions are visible, and monitoring covers change.

Hold the conclusion when evidence comes only from policy or screenshots, the denominator depends on the label being present, transformations are ignored, inaccessible labels count as successful, indeterminate cases disappear, or a disclosure is described as proof of output quality.

Two Prune provides realistic AI-enabled work assessments and evidence about observed participant performance. It may assess whether a participant notices disclosure, checks provenance, verifies content, and communicates uncertainty in a bounded task. It does not certify labeling controls, determine legal sufficiency, or make employment decisions.

Sources

NIST: Reducing Risks Posed by Synthetic Content

  • NIST AI RMF: Accountable and Transparent

Sources: [1] [2]

Sources

  1. 1.NIST: Reducing Risks Posed by Synthetic Content · National Institute of Standards and Technology
  2. 2.NIST AI RMF: Accountable and Transparent · National Institute of Standards and Technology