How to develop and assess provenance tracking in AI-enabled work

Track derivation, not just links

The World Wide Web Consortium's PROV model describes provenance through entities, activities, and agents. It connects an output to what generated it, what it used, what it was derived from, and who or what was associated with the activity. NIST's AI Risk Management Framework similarly emphasizes documented context, data selection, third-party components, testing, limitations, roles, and human oversight.

A URL in a footnote is useful, but it is not a complete provenance record. The linked page may change. A chart may derive from a downloaded table rather than the current page. An AI summary may combine three sources while citing only one. A spreadsheet value may have been normalized, filtered, or manually corrected. Provenance must preserve the chain that explains the current artifact.

The required detail should match the task. A short internal briefing may need source, retrieval date, locator, and transformation note. A high-impact calculation may need input versions, formulas, tools, approvals, and reproducible intermediate files. The goal is not maximal metadata. It is enough lineage to verify, update, or challenge the work.

Build a LINEAGE record

For each material output, create a LINEAGE record:

| Element | Provenance question | Evidence artifact | | --- | --- | --- | | Locator | Which exact source entity and version was used? | Stable identifier, title, date, page, cell, or section | | Input | What source content entered the work? | Quotation, extracted value, or retained snapshot | | Nature of change | Was it copied, filtered, summarized, calculated, translated, or inferred? | Transformation description and rule | | Executor | Which person, model, script, or tool performed the activity? | Agent and tool/version record | | New entity | What output or intermediate artifact was produced? | File, row, chart, claim, or draft identifier | | Integrity | Can the input and output be matched to the recorded versions? | Hash, version ID, or reproducibility check | | Approval | Who reviewed or accepted the transformation, and when? | Reviewer, disposition, and timestamp | | Gap | Where is the chain incomplete or dependent on a mutable source? | Limitation and recovery action |

Use identifiers that survive renaming. A filename such as final-v2-revised.xlsx is not reliable version evidence. Where hashes are practical, bind the record to exact content. Where a system supplies immutable version IDs, retain them. For a dynamic web source, capture the retrieval time and the specific passage used.

When AI contributes, separate the source entity from the model activity. The model is not the origin of facts quoted from a policy or dataset. Record the prompt or task instruction at the level needed to reproduce the transformation, the model or tool used, the source context supplied, and any human corrections that created the final entity.

Worked example: a synthetic board metric

Imagine a synthetic finance analyst preparing a quarterly board chart for customer-retention performance. The chart appears in a slide deck and shows retention increasing from 88 to 91 percent. This example is original and does not describe a Two Prune customer.

The analyst receives a dashboard screenshot, a CSV export, and an AI-generated narrative from a colleague. The screenshot has no visible filter state. The CSV filename says “latest,” and the narrative cites the dashboard home page. A simple source list would preserve three links but not explain the number.

The analyst builds a LINEAGE record:

The analyst discovers that the 88 percent comparison value cannot be placed on the same basis without rerunning the prior quarter. The AI narrative's claim of a three-point improvement is therefore held. The current-quarter 91 percent can still be reported with its definition, while the trend statement becomes pending.

The analyst reruns the earlier period using definition 3.2, producing 89.4 percent. The final chart shows 89.4 to 91.0 percent, includes a definition note, and links to a compact lineage manifest. If the metric definition changes again, the team can identify which periods and artifacts require recalculation.

The value of provenance is not bureaucratic completeness. It is the ability to discover that two visually comparable numbers came from different transformations before a decision is made.

• Locator: warehouse export job R-2841, completed 3 September, dataset version 7, plus metric definition version 3.2;
• Input: active accounts and retained accounts for the quarter, with test accounts excluded;
• Nature of change: filter to the approved customer population, deduplicate account merges, divide retained by eligible active accounts, and round to one decimal place;
• Executor: SQL query retentionboardq3.sql run by the analyst, followed by a chart created in the approved spreadsheet template;
• New entity: chart C-17 and the sentence describing the quarter-on-quarter movement;
• Integrity: hashes for the CSV, query, and exported chart data table;
• Approval: finance controller reviewed the definition and reconciliation on 5 September;
• Gap: the previous quarter used metric definition 3.1, which treated paused accounts differently.

Practice with a broken chain

Give a learner a short report containing five claims, a chart, and a table. Supply the apparent sources plus a folder of intermediate files. Include one source that changed after retrieval, one chart generated from the wrong CSV version, one AI paragraph that blends two sources, and one manually corrected value with no note.

Ask the learner to create LINEAGE records for the decision-critical artifacts. They should identify which chains are complete, which can be reconstructed, and which outputs must be qualified or withdrawn. Then provide a corrected input and ask which downstream entities are affected.

Useful practice behaviors include:

• identifying the exact source entity and version;
• preserving the input passage or value actually used;
• recording transformations rather than only final citations;
• distinguishing people, models, scripts, and source owners;
• binding critical artifacts to hashes or stable version IDs;
• tracking human corrections and approval decisions;
• propagating a changed input through dependent outputs;
• making incomplete lineage visible in the final communication.

Assess observable provenance behavior

In an assessment, include a multi-step artifact built from documents, data, and AI assistance. Observe whether the participant:

Do not score provenance by metadata volume. A concise chain that lets a reviewer reproduce the critical number is stronger than a large log with no connection between inputs and outputs. Provenance quality and factual correctness are related but distinct: a perfectly traced calculation can still apply the wrong rule.

• records precise source and version identifiers;
• distinguishes original evidence from intermediate and final artifacts;
• captures filtering, calculation, summarization, and inference steps;
• identifies who or what performed each material transformation;
• preserves the relationship between AI output and supplied sources;
• detects mutable links, stale files, and ambiguous “latest” versions;
• uses integrity checks proportionate to the risk;
• traces corrections into dependent claims, tables, and charts;
• states lineage gaps rather than presenting unsupported reproducibility;
• gives reviewers a practical route back to the evidence.

Verify with lineage replay

Select one high-impact output and give its LINEAGE record to a reviewer who did not create it. The reviewer should retrieve the recorded inputs, apply the documented transformation, and reproduce the result within the stated rounding or interpretation rule.

Next, change or revoke one input. Use the dependency links to identify every output that must be recalculated, relabeled, or held. If the team cannot find the downstream artifacts, the lineage record is incomplete even if the original result can be reproduced.

Finally, inspect whether approvals attach to the exact artifact reviewed. A sign-off on an earlier spreadsheet should not silently carry over to a regenerated chart or revised narrative. Rebind approval when content changes materially.

Provenance tracking supports inspection and reproducibility within the recorded chain. It does not prove that the originating source is true or that the chosen method is appropriate. In a Two Prune assessment, the LINEAGE records, source navigation, AI interactions, integrity checks, correction propagation, revisions, and final artifact can contribute bounded evidence about Evidence Navigation, AI Orchestration, Verification & Risk Control, and Communication & Delivery. They do not make an employment decision.

Sources

W3C PROV Ontology Recommendation

• NIST AI Risk Management Framework Core

Sources: [1] [2]