What an agent did must remain reconstructable

An agent is not controllable merely because it writes a convincing final report. Control requires its actual path through sources, decisions, tools and state changes to remain verifiable.

A long agentic session often ends with a reassuring summary: research completed, files updated, tests passed, next steps prepared. Such reports are useful, but they are self-reports. A file may be missing, a source misread or a tool may have produced an unexpected side effect while the same agent still narrates a smooth success story.

Responsible agentic work therefore needs more than memory and closing prose. It needs a reconstructable trail. What was the initial state? Which objective applied? Which sources were actually read? Which parameters reached a tool? What changed? Which checks ran? Who authorised continuation? Only when these questions can be answered later does a conversation become a controllable process.

Status is a snapshot, not proof

A project table of open, completed and blocked work can stabilise long sessions. It condenses valid state and supports handoff. Yet it usually shows only what the system claims about state. It does not necessarily explain how that state arose.

A task may be marked complete when only a draft exists. A source may be called reviewed when the agent read a summary. A test may be „passed" against stale data. Status remains useful, but must connect to events and artifacts.

The stronger statement is not „Task 17 complete" but: „Artifact version 4 was produced from sources A and B; test set C ran against revision D; two criteria passed; one exception was documented and accepted by role E." Status then becomes verifiable.

Observability, audit trails and provenance answer different questions

Observability exposes runtime behaviour. Traces represent connected operations, logs individual events and metrics aggregated measures such as latency or error rate. Audit trails focus on control-relevant actions: approvals, access, changes, publication and exceptions. Provenance explains creation and derivation: which inputs, activities and responsible actors produced an output?

These layers complement one another. A trace may show a search, three tool calls and a file write. The audit trail records that the write was authorised. Provenance connects the new file to consulted sources and transformation.

Not every debug detail belongs permanently in an audit trail, and not every audit event needs full prompt content. Clean architecture separates signals, defines shared identifiers and enables correlation.

An agentic trace needs more than a model name

A useful trace starts with a stable run ID and divides work into spans. A span may represent planning, retrieval, tool execution, evaluation or human approval. Parent-child relationships show which operation triggered a sub-process.

An event needs timestamp, actor, task, operation, input references, output references, status, duration and error. Tool calls also need tool version, validated parameters, permission decision and side effect. Retrieval records query, data-source ID, selected documents and relevant versions. Evaluation records metric, test version, result and explanation.

OpenTelemetry is developing common semantic attributes for generative AI. Conventions do not solve every governance problem, but provide shared language across libraries and platforms. Custom fields need the same discipline: clear meaning, stable type, documented scope and versioning.

Separate claims, events and artifacts

Agents frequently say, „I validated the file." That sentence is a claim. The validation event is the operation that actually ran. The validation report is the resulting artifact. Reliable traceability requires all three to be connected.

This prevents language from replacing technical reality. A model may say „saved" after a failed write. Status should update only when the host observes the new state and confirms an event or artifact.

Important claims follow the same rule. A recommendation points to evidence cards containing source, excerpt or dataset reference, date, scope and verification. The article or report cites the cards rather than an invisible model memory.

State changes need before and after

A read-only agent creates different risk from a writing agent. When files, database records, permissions or external systems change, the audit trail captures the mutation: target object, previous state or version, intended change, actual outcome and rollback path.

Large or sensitive objects need not be duplicated in full. Version, hash, diff, location and protected metadata may be enough. The record must allow a later reviewer to establish which change occurred under which authority.

Reversible and irreversible work use different gates. A draft can be created in isolation. Publication, deletion or transaction needs a checkpoint. Approval is tied to scope and expiry; permission for object A must not silently authorise object B.

Cost and time are behaviour

Agent quality is often judged only by final output. Yet a system may produce the same answer with ten calls or ten thousand. Unbounded loops, repeated retrieval and failed tool retries are operational defects even when the final text is usable.

Observability therefore captures latency, model and tool calls, input and output volume, cache use, retries and estimated cost. Values are viewed per step, task and run. Budgets are control limits, not just report columns. Exceeding a time, cost or step limit pauses or escalates the run.

Aggregated metrics reveal patterns: Which role repeats work? Where do delays occur? Which tool fails most often? Individual traces then explain outliers.

Errors must retain their cause

An agent that replans successfully after failure is valuable. The successful retry should not erase the first error. Otherwise the material needed for improvement and risk analysis disappears.

An error event records category, affected step, input, tool response, visible side effect and selected reaction. It distinguishes timeout, denied permission, schema error, empty retrieval, conflicting evidence and professional uncertainty. „A problem occurred" is not diagnosable.

Retries link back to the original error. Strategy changes carry rationale. Repeated failure may trigger a circuit breaker and stop the run. Autonomous persistence must not become automated damage.

Complete logging can become a risk

Prompts, tool parameters and responses may contain personal data, trade secrets, credentials or confidential documents. Blind capture creates a second sensitive database and expands the attack surface.

Logging therefore follows data minimisation. Default records use references, categories, hashes and necessary metadata. Full content is captured only when the purpose demands it, authority is established and access tightly controlled. Secrets are filtered, sensitive fields masked and tenants separated.

Retention is tiered. Short-lived debug data may disappear after days while control-relevant approvals are needed longer. Deletion rules, access rights and purpose limitation apply to observability data too. An audit trail is not permission for unlimited collection.

Tamper resistance and accountability

An audit trail loses value if the acting agent can rewrite it. Critical events are stored append-only or protected through hash chains, signatures and versioned write-protected storage. The agent may propose event content, but the runtime creates the authoritative record.

Accountability needs stable identities for agent, model configuration, skill, tool and human role. „The model" is too vague. Version, configuration, permission context and executed code matter. In multi-agent systems, each message carries sender, recipient, task ID and trust state.

Tamper resistance does not prove that every entry is true. It proves first that a record has not changed since it was written. Plausibility checks and independent evidence remain necessary.

Dashboards are an entry point, not control

A dashboard can present state, errors, cost and pending approvals. It should not show only green indicators. Users must be able to move from metric to trace, trace to event and event to artifact or source.

Good views expose uncertainty and missing telemetry. An uninstrumented tool call is not a successful tool call. A trace gap is marked visibly. For critical processes, incomplete observability may itself block continuation.

People need decision-ready views: What changed? Which evidence is missing? Which exception was used? What will continuation cost? How can the action be rolled back? Raw logs remain available for diagnosis but are not mistaken for a user interface.

Evaluation connects traces to quality

Observability says what happened; evaluation judges whether it was good enough. A test result references run, artifact version, dataset and criteria. Operational failures become new evaluation cases. A new agent version is tested against stored traces or reproducible scenarios.

NIST emphasises risk-based measurement and treatment across the lifecycle. For agents, complete trajectories matter — not only model responses. Did the agent choose the right source, respect authority, avoid unnecessary steps, recognise uncertainty and stop safely?

Observability becomes learning infrastructure. It does not create automatic truth, but supplies evidence from which teams detect failure, improve controls and prevent regressions.

Worksheet: Build a reconstructable agent run

Choose a workflow containing retrieval, a tool action and verification.

1. Define run ID, task ID, agent identity and artifact versions.

2. Draw trace and spans from objective to handoff.

3. Define events for retrieval, tool call, mutation, error, retry, evaluation and approval.

4. Connect every important claim to an evidence reference.

5. Capture before/after, diff or hash for every state change.

6. Define metrics and budgets for steps, time, cost and errors.

7. Mark sensitive fields, masking, access and retention.

8. Simulate tool failure and log tampering; test stop, escalation and reconstruction.

All materials to download — the topic overview and the worksheet:

Scope: Observability, audit trails and provenance overlap but are not identical. Telemetry exposes runtime behaviour; an audit trail records control-relevant events immutably and with accountability; provenance describes how artifacts and claims were produced. Complete logging is neither automatically useful nor privacy-compliant. Content, detail, access and retention must follow purpose and risk.