RAG is not a chatbot with attached PDFs

A document chat may look like retrieval-augmented generation: PDFs are added, a question is asked and a fluent answer appears. Yet the implementation may only provide temporary file context. It may lack a governed knowledge corpus, explicit search strategy, stable source references, separate quality measurements and an update process. The tool can still be useful; that alone does not make it a defensible RAG architecture.

RAG connects parametric model knowledge with an external, replaceable knowledge source. The foundational research describes a generator combined with non-parametric memory. In practice this is not one feature but a pipeline. Quality depends on document preparation, indexing, retrieval, context selection, answer logic and evidence—not only on the language model.

Four stages instead of a file attachment

An understandable RAG pipeline has four core stages. Retrieval finds candidates for a query. Context Assembly orders, filters and limits the passages. Synthesis produces an answer using that context. Source Binding links claims to verifiable locations.

Every stage can fail independently. Retrieval can miss the decisive passage. Context assembly can cut off an exception. Generation can add a plausible invention. Citation logic can display a topically related source that does not support the actual claim. A polished interface hides these differences; professional engineering exposes them.

The knowledge corpus exists before the first question

Before chat comes the ingestion pipeline. Documents are acquired, checked for readability, parsed, structured, segmented, enriched with metadata and indexed. Version, validity, owner, language, permission and provenance must survive. A scan without OCR, a table without column relations or a manual without version status does not become reliable merely by receiving an embedding.

At query time, the complete archive is not copied into the context window. The retriever creates a bounded candidate set using lexical, semantic, hybrid or multi-stage search. Results may then be reranked, filtered by permission and arranged within a context budget. Only then does answer generation begin.

A vector index is a search component, not a knowledge guarantee

Embeddings represent content numerically; similarity search finds nearby vectors. This is powerful, but proximity does not establish professional relevance, currency or authority. Dense Passage Retrieval demonstrates retrieval with dense representations. FAISS demonstrates efficient similarity search over large vector collections. Neither decides which source is binding in a particular organisation.

Metadata and policy therefore remain essential. A result can be semantically close while expired, unapproved or restricted for the requesting role. Retrieval must enforce those conditions instead of leaving them to a language model after the fact.

RAG changes knowledge without fully retraining a model

One advantage is that the external knowledge corpus can be replaced. Documents may be added, blocked or re-indexed without fully retraining the base model. This can make updating and provenance easier, but it does not ensure immediate propagation. Caches, replicas, old indexes and delayed pipelines also require control.

RAG is therefore useful when answers should depend on a defined, changing corpus. A simpler design may be better for tasks without knowledge retrieval or for a very small set of stable rules. Architecture begins with the problem, not with a desire for a vector database.

RAG moves the problem

Without retrieval, the central questions concern what the model learned and how reliably it can use that knowledge. RAG adds more: does the corpus contain the required information? Is it valid, accessible and correctly described? Was the query interpreted properly? Did the retriever find the relevant passage? Was it ranked high enough, included in the context window and used faithfully?

RAG reduces some risks while adding a complex system surface. The foundational NeurIPS work combined parametric memory with retrievable non-parametric memory and reported more specific and factual language than a parametric-only comparison. That result does not imply that every modern pipeline is reliable. Corpus, retriever, ranking, prompt, model and evaluation differ substantially.

A useful model separates six controls: ingest knowledge, constrain candidates, retrieve evidence, rank evidence, assemble context and generate an answer. Each can preserve or destroy quality.

Similarity is neither truth nor authority

Vector search roughly identifies representations close to a query. It does not automatically establish which source is true, current, approved or authorised for the requester. A detailed obsolete draft may be semantically closer than a brief current decision. A widely repeated claim may be easier to retrieve than its correction.

Retrieval therefore needs an admissible candidate space. Tenant, permission, protection class, validity period, status, language, region and permitted purpose are filtered before or enforceably within search. Security rules cannot live only in the prompt. The model should never receive content it is not authorised to process.

Hybrid search can then combine lexical and semantic signals. Exact identifiers, names, clauses and product codes often benefit from keyword retrieval; paraphrases benefit from dense representations. A reranker can score candidates more precisely for the query. These tools improve selection but do not replace authority, status or validity logic.

Query transformation can help and distort

Systems rewrite user questions, generate subquestions, expand synonyms or create hypothetical answers as retrieval anchors. These techniques can increase recall. They can also shift meaning, remove constraints or turn an open question into an unsupported assumption.

Preserve the original query. Record every rewrite as a derived object and compare it in tests. Critical constraints—time, person, product, jurisdiction—must survive transformation. For complex questions, multiple searches followed by evidence synthesis are often safer than treating one „optimised" query as authoritative.