Data needs a lifecycle

Many data projects think carefully about ingestion and use, but barely consider what follows. Documents are uploaded, tables merged, conversations indexed and outputs retained. The collection grows with every step. What is often missing is a system that determines when an object becomes active, who reviews it, what supersedes it and how it is responsibly retired.

A lifecycle is not a circular graphic for presentations. It is a state model with accountabilities, rules, events and testable transitions. That is what keeps a knowledge space current, legitimately usable and controllable by AI systems.

The lifecycle begins before capture

The first question is not where to store a file. It is why the information is needed. Purpose, intended use, affected people, protection needs and evidence requirements determine which data should be captured and what quality is necessary.

Collecting everything because it may become useful merely defers the decision. Unknown copies, conflicting versions and ownerless stores follow. Personal data is also subject to context-specific legal principles, including purpose limitation, minimisation and storage limitation. Other information may be governed by statutory, contractual or professional retention duties.

At entry, an object should therefore receive at least purpose, source, accountable owner, protection class, status, validity period, review date and intended disposition rule. Missing values create lifecycle debt from the first day.

States and transitions matter more than phase names

Labels such as create, store, use, archive and delete provide orientation, but real data does not move in one direction. An archived record may be restored for an audit. A source may be withdrawn, corrected and republished. A draft may be rejected before becoming active.

A state model is more precise. A knowledge object might be draft, in_review, approved, active, restricted, superseded, archived, legal_hold or disposed. Every transition has a trigger, authorised role, preconditions, event record and downstream effects.

object_id: contract-guide-v4
state: active
valid_from: 2026-07-01
review_due: 2027-01-15
owner_role: legal_content_owner
retention_class: business_guidance
supersedes: contract-guide-v3
on_expiry: block_retrieval_and_review

Value comes from enforcement. When review_due passes, the system should not merely send an email and continue unchanged. Depending on risk, it labels the object, removes it from default retrieval, requests review or continues only with a visible limitation.

Raw data, curated data and active evidence are different

A robust architecture separates layers. The raw zone preserves the original capture unchanged and protected. The curated zone contains cleaned, normalised or enriched versions with documented derivation. The active knowledge space includes only sources approved for a purpose, period and audience. The archive preserves selected historical objects and evidence without injecting them into every AI request.

This prevents two common failures. A correction is not silently written over the original, and "findable" is not confused with "currently usable". Historical records may remain valuable without overriding a current decision out of context.

Quality and approval gates sit between layers. They assess identity, format, completeness, provenance, rights, protection class, version and purpose. A gate is a recorded decision, not merely a script that exited successfully.

Every use creates new lifecycle objects

AI systems do not only consume data. They produce chunks, embeddings, summaries, classifications, prompts, tool results, logs and answers. Each artefact has its own purpose, sensitivity and lifecycle.

A document may be deleted while its chunks remain in a vector index. A false figure may disappear from the knowledge base but survive in a stored summary. Personal information may leave the interface while remaining in debugging logs. Provenance must therefore expose derivations in both directions.

For retrieval-augmented generation, the chain might be original → extraction → curated document → chunk → embedding → index entry → retrieved evidence → answer. A source-state change triggers defined actions along this chain: block, re-extract, re-embed, invalidate the index, review dependent summaries and flag affected outputs.

Not every output should be retained indefinitely. Some run data is useful only for short-term debugging; other evidence may be needed longer for audit, safety or professional accountability. Purpose and risk — not available storage — drive the decision.

Retention asks two questions: how long and in which state?

"Seven years" is not a complete retention rule. It needs a starting event, exceptions, accountable role and final action. Does the period begin at creation, contract termination, last modification or closure of a case? What happens during litigation or investigation?

Retention also does not imply that data stays active in AI context for the whole period. An object may be protected in an archive for evidence while excluded from search, training and generation. Access, encryption, integrity control and recoverability should match the archive function.

A legal_hold or comparable preservation status may temporarily suspend ordinary disposal. It must be justified, authorised, reviewed and eventually released. Permanent exceptions indicate weak governance.

Deletion is a chain, not a button

Removing a database row does not automatically end the lifecycle. Copies may remain in caches, search indexes, exports, data lakes, analytical tables, logs, backups and user devices. Derived artefacts may preserve part of the content.

Disposition therefore needs scope. The system discovers dependent objects, applies the intended method, records success and failure and checks residual stores. Media sanitisation is a separate technical discipline: NIST SP 800-88 Rev. 2 frames it as rendering access to target data infeasible for a given level of effort, with methods selected according to sensitivity and context.