RAG is not a truth machine

Retrieval-augmented generation appears to solve a familiar problem elegantly. Instead of relying only on parametric model knowledge, the system searches an external collection and provides relevant material as context. Sources can be updated, internal documents included and answers linked to evidence.

This is a major improvement, but it is not a truth mechanism. A RAG system can retrieve the wrong source, split the right source badly, miss a valid passage, rank obsolete content higher, expose unauthorised material or distort evidence during generation. Quality is created along an evidence chain, not by one model call.

RAG moves the problem

Without retrieval, the central questions concern what the model learned and how reliably it can use that knowledge. RAG adds more: does the corpus contain the required information? Is it valid, accessible and correctly described? Was the query interpreted properly? Did the retriever find the relevant passage? Was it ranked high enough, included in the context window and used faithfully?

RAG reduces some risks while adding a complex system surface. The foundational NeurIPS work combined parametric memory with retrievable non-parametric memory and reported more specific and factual language than a parametric-only comparison. That result does not imply that every modern pipeline is reliable. Corpus, retriever, ranking, prompt, model and evaluation differ substantially.

A useful model separates six controls: ingest knowledge, constrain candidates, retrieve evidence, rank evidence, assemble context and generate an answer. Each can preserve or destroy quality.

Similarity is neither truth nor authority

Vector search roughly identifies representations close to a query. It does not automatically establish which source is true, current, approved or authorised for the requester. A detailed obsolete draft may be semantically closer than a brief current decision. A widely repeated claim may be easier to retrieve than its correction.

Retrieval therefore needs an admissible candidate space. Tenant, permission, protection class, validity period, status, language, region and permitted purpose are filtered before or enforceably within search. Security rules cannot live only in the prompt. The model should never receive content it is not authorised to process.

Hybrid search can then combine lexical and semantic signals. Exact identifiers, names, clauses and product codes often benefit from keyword retrieval; paraphrases benefit from dense representations. A reranker can score candidates more precisely for the query. These tools improve selection but do not replace authority, status or validity logic.

Chunking determines what can be seen

Documents are divided because entire files rarely fit efficient indexing and generation. Chunks that are too small lose definitions, exceptions or temporal scope. Chunks that are too large dilute relevant signal and consume context. Fixed character windows can separate a heading from its paragraph, a table header from values or a rule from its exception.

Chunking should respect structure: headings, paragraphs, lists, tables, pages, speakers and domain units. Every chunk inherits document ID, position, version, status, language, permission and provenance. Overlap can preserve context, but creates duplicates and must not inflate retrieval measurements.

Cross-section relations are especially difficult. A definition may appear at the beginning, an exception three pages later and the effective date in an annex. No single chunk contains the answer. Parent-child retrieval, neighbourhood expansion or multi-stage retrieval can help. Such cases must exist in the evaluation set.

Query transformation can help and distort

Systems rewrite user questions, generate subquestions, expand synonyms or create hypothetical answers as retrieval anchors. These techniques can increase recall. They can also shift meaning, remove constraints or turn an open question into an unsupported assumption.

Preserve the original query. Record every rewrite as a derived object and compare it in tests. Critical constraints — time, person, product, jurisdiction — must survive transformation. For complex questions, multiple searches followed by evidence synthesis are often safer than treating one "optimised" query as authoritative.

Conversation history can also interfere. A pronoun may need context, but previous assumptions should not silently control a new search. Query rewriting is a controlled interpretation step, not an invisible convenience filter.

Retrieved evidence is not yet a faithful answer

Even with the right passage in context, the model may ignore, miscombine or embellish it. Fluent text can be stronger than its source: "may" becomes "must", an example becomes a universal rule, correlation becomes causation.

An evidence contract narrows the task. Material claims point to concrete passages. Contradictions remain visible. The system distinguishes direct evidence, permitted inference and unsupported interpretation. If evidence is insufficient, it lowers the claim, asks a question or abstains.

Abstention is a quality capability. A system that always answers optimises appearance rather than reliability. Thresholds follow risk: cautious hypotheses may support ideation; consequential decisions require stronger evidence and often human review.

Citation is a separate verification task

A citation can exist and still be wrong. The passage may mention a topic without supporting the assigned claim. An answer may cite one sentence correctly and leave other material claims unsupported. Three questions therefore require separate evaluation:

1 · Citation correctness: does the passage entail or support the associated claim?

2 · Citation completeness: are all material verifiable claims supported?

3 · Source quality: is the source appropriate for purpose, time and scope?

Citations should identify the used version and, where possible, a stable passage. A home page or entire PDF makes verification harder. The system must not present generated prose as a quotation. Direct quotation, paraphrase and synthesis remain distinguishable.

Provenance connects answer, run, retrieved chunks, index version, extraction and original. This allows error localisation and impact analysis after source withdrawal.

The knowledge base is an attack surface

External and user-uploaded documents can contain instructions that models interpret as commands. OWASP describes this as indirect prompt injection: content from files or websites changes system behaviour. RAG and fine-tuning do not fully eliminate the risk.

Retrieved content is therefore treated as data, not privileged instruction. System policy and tool permissions remain separate. Untrusted sources are identified, inputs and outputs inspected, tools follow least privilege and high-impact actions require human approval. A document must never gain authority over tools merely because it is semantically relevant.

The index itself can be manipulated. Many similar documents can crowd out rankings; poisoned text can spread false relations or hidden instructions. Ingestion needs provenance, approval, deduplication, quotas, anomaly detection and controlled update paths.

Evaluate retrieval and generation separately

A single judgement that the "answer was good" does not locate failure. Evaluation begins with the corpus: does it cover the questions and contain valid versions? Retrieval is then tested: does relevant evidence appear in the top-k, at which position, and are prohibited or obsolete sources excluded?

Generation evaluation asks whether the answer is supported by supplied context, professionally correct, appropriately uncertain, citation-complete and capable of abstaining when the question is unanswerable.

A defensible test set contains ordinary questions, rare terms, ambiguity, multi-hop cases, old versions, contradictions, hard negatives, missing answers and attacks. It measures critical error classes and affected groups, not only averages.

Retrieval metrics aid diagnosis. Recall@k asks whether relevant evidence appears among the first k results. Precision measures the relevant share. MRR weights the rank of the first relevant result; nDCG accounts for graded relevance and ordering. None replaces professional relevance judgements. BEIR also demonstrates that retrieval architectures vary across domains and that robust baselines matter.

Production quality requires observability

After deployment, the corpus changes. Documents are added, embedding models replaced, chunking adjusted and user questions evolve. Index, schema, retriever, reranker, prompt and model versions belong in the answer run.

Monitoring tracks empty retrieval, low scores, unusual source concentration, abstention, authorisation failures, latency, cost and user corrections. Changes run against regression tests first. Better average performance must not hide degradation in critical questions or access protection.

The most important measure is often not how convincing the system sounds. It is how reliably it finds appropriate evidence, recognises boundaries and supports inspection. RAG is strongest when it exposes uncertainty and makes the bridge from question to source to answer visible.

Worksheet: Decompose a RAG answer

Choose at least ten documents and create eight test questions.

1. Qualify the corpus. Label valid, obsolete, restricted, contradictory and insufficient sources.

2. Compare chunking. Create two segmentation strategies and test definitions, exceptions, tables and multi-hop questions.

3. Test retrieval. Compare lexical, semantic and hybrid retrieval with a reranker.

4. Enforce permissions. Ensure restricted content never reaches model context, even with perfect similarity.

5. Apply an evidence contract. Map each material claim to a passage and label inference or uncertainty.

6. Test abstention. Add two unanswerable questions and define the expected response.

7. Simulate an attack. Place an indirect instruction in a test document and inspect tool and response behaviour.

8. Locate failures. Assign every error to corpus, chunking, query, retrieval, ranking, context, generation or citation.

All materials to download — the topic overview and the worksheet:

Scope: RAG is an architecture family, not a single quality promise. Retrieval can make answers more current, specific and evidentially supported, but it does not guarantee truth, completeness, legality or security. Metrics such as Recall@k, Precision, MRR and nDCG require interpretation against the dataset, relevance judgements and use case.