Reliable AI requires provenance

AI can produce a fluent and detailed answer that is unusable. If its basis is unknown, an error cannot be corrected and a decision cannot be defended. The problem grows when systems combine documents, databases, search indexes, APIs and tool calls.

Without provenance, nobody can tell whether a claim came from an approved policy, an obsolete draft, an automatically generated summary or an unsecured tool result. A bibliography helps, but it is not a complete provenance record.

Provenance, citation, lineage and audit log

Four related concepts serve different purposes. A citation points to evidence for a claim. Data lineage describes how data flows through systems and transformations. Provenance records the origin and creation history of an object, including agents and processing steps. An audit log records events for later inspection.

A defensible system connects all four. The answer cites evidence. Lineage shows how the dataset or index was produced. Provenance links source, transformation and accountability. The log records when a process ran and which permissions were active.

A simple model: entity, activity and agent

The W3C PROV model centres on three building blocks:

1 · Entity: a document, dataset, segment, model artefact, report or concrete output in a particular state.

2 · Activity: extraction, cleaning, translation, summarisation, indexing, approval or model inference.

3 · Agent: a person, organisation, software component or accountable role participating in an activity.

Relations carry the history: an output was generated by an activity; an activity used inputs; a new version was derived from an earlier one; an agent was associated with or accountable for the work.

Many projects can implement this compactly. Each object receives a persistent identifier, source reference, timestamp, version, status and accountable role. Every transformation links input, method, parameters, tool version and output. Approval is captured as an event rather than hidden in a filename.

entity_id: policy-summary-v3
derived_from: policy-source-v7
activity: summarization
tool_version: summarizer-2.4
created_at: 2026-07-16T10:42:00Z
status: approved
approved_by_role: policy_owner
supersedes: policy-summary-v2

Preserve the original

A provenance chain needs a stable beginning. The original document, event or measurement remains immutable. Cleaning, conversion and enrichment create new versioned objects. Later investigation can then determine whether an error existed in the source or appeared during processing.

Content hashes can help detect byte changes, but they are not truth certificates. A hash does not prove that a source is correct, lawfully used or created by the claimed author. Trusted identity, signatures, permissions and professional approval remain separate controls.

Versioning needs professional meaning

"Final", "final_new" and "really_final" are not status models. A version number shows sequence but not authority. The system must also record what changed, why, when it becomes effective and which earlier version it supersedes.

A useful lifecycle distinguishes draft, in review, approved, superseded, withdrawn and archived. Retrieval must be able to filter these states. Semantic similarity may otherwise return the closest passage rather than the currently valid one.

Supersession relations enable impact analysis. When a source is withdrawn, teams should identify dependent segments, summaries, indexes and outputs. Provenance therefore supports correction as well as retrospective inspection.

The chain continues through the AI run

Knowing the original document is not enough. The observable path to an answer should be reproducible: request and approved prompt, retrieved documents and segments, active filters and permissions, knowledge-base and index versions, model and tool versions, and human intervention.

This does not require storing every internal activation or hidden chain of thought. Operational traceability covers inputs, configuration, retrieved evidence, tool results, decisions, output and approval. It should diagnose errors without logging unnecessary sensitive content.

This is especially important for agents that search, execute code, modify files or trigger external actions. Each tool call needs identity, authorisation, input, result, status and correlation to its parent task. A later reviewer must be able to identify which step caused a change.

Provenance itself requires protection

Logs may contain personal data, confidential passages, access tokens or internal decisions. Completeness does not justify unlimited collection. Provenance must follow minimisation, purpose limitation, role-based access, retention limits and tamper-resistant storage.

Secrets do not belong in plaintext logs. Where possible, record references, persistent identifiers and redacted parameters. Access to provenance records should itself be logged. Even a complete lineage loses value if it can be altered unnoticed.

What provenance does not prove

A complete chain can faithfully document the history of false information. Provenance answers where an object came from, how it changed and who participated. It does not automatically answer whether a claim is true, the collection method was suitable or the data may lawfully be used for the present purpose.

Source criticism, data-quality assessment, legal basis and professional validation therefore remain distinct controls. Human approval is not an infallible truth certificate either; it records an accountable decision at a point in time. Strong systems preserve these boundaries instead of inferring more assurance from metadata than the evidence supports.

Provenance can also exist at different levels of detail. A low-risk internal summary may only require document and model version. A consequential automated decision may require feature-level lineage, applied rules, tool calls and approvals. Granularity follows risk and the degree of reconstruction the organisation must support.

Test the provenance model

A schema alone does not guarantee traceability. Tests determine whether every derivative reaches a source, transformations record tool and parameter versions, and approved objects identify accountability. Negative tests simulate missing sources, broken links, withdrawn versions and unauthorised access.

A backward test for an AI answer should traverse output → run → retrieved segments → index version → extracted document → immutable original. A forward test asks which dependent artefacts and outputs are affected when a source is withdrawn.

Provenance is not a decorative source block. It is operational infrastructure for questions, reproduction, withdrawal, updates and audit. Most importantly, it prevents linguistic confidence from being confused with factual validity.

Worksheet: Reconstruct a provenance chain

Choose a published figure, claim or AI answer and document its complete path.

1. Identify the source object. Record persistent identifier, origin, creator, time, protection class and immutable original.

2. Model transformations. Document at least two activities with input, method, parameter or tool version and output.

3. Clarify version and status. Specify the valid version, approval, validity period and superseded version.

4. Reconstruct the answer run. Record segments, filters, index, model and tool versions and human intervention.

5. Create a second version. Change one claim with justification and document derivation and supersession.

6. Assess impact. Determine which downstream artefacts require review or blocking if the source is withdrawn.

All materials to download — the topic overview and the worksheet:

Scope: Provenance documents creation and processing history. It does not automatically prove truth, legality or professional correctness. A hash can support byte integrity but does not by itself establish authorship or factual validity. Concrete record-keeping duties depend on role, system classification and context.