Part IV · Ch. 31 — RAG (Retrieval-Augmented Generation)
Hybrid · Language · LLM + vector search. Retrieves relevant documents and injects them into the LLM's context to answer with sources. Card:
../02-types-of-ai/06-hybrid-neuro-symbolic.kmd.
🎨 Figure
F-IV.31.0— Question → search → grounded answer. Brief: the question becomes an embedding → search in a vector index → retrieved passages enter the LLM prompt → answer with citations.
1. Definition and short history
Combines a retriever (semantic search) with a generator (LLM) to ground answers in external, up-to-date knowledge. It became the standard for enterprise LLM applications (2023+) by reducing hallucination and citing sources.
2. Foundations
- Information / retrieval theory — relevance, ranking, BM25.
- Linear algebra — embeddings and cosine similarity; ANN.
- Probability — re-ranking and relevance calibration.
- Linguistics — semantic chunking, meaning matching.
3. Algorithms and architectures
- Indexing — documents → chunks → embeddings → vector index
(HNSW/IVF).
- Retrieval — similarity search (dense) + lexical (BM25) → hybrid.
- Re-ranking — cross-encoder refines the top-k.
- Generation — LLM answers conditioned on the passages; cites sources (grounding).
- Variants — GraphRAG, agentic RAG, iterative retrieval.
4. Inputs
- Hardware: GPU (embeddingsLLM); CPUindex for search.
- Data: knowledge corpus (docs, wikis, databases); continuous updating.
- Data structures: vector index (HNSW), embeddings, metadata.
- Systems: vector DB, ingestion pipeline, cache, observability.
5. Specialized life cycle
| Stage | Specialization |
|---|---|
| 0 Problem | Knowledge domain, freshness required, need for citation |
| 1 Data | Doc ingestion, chunking, embeddings, deduplication, permissions |
| 2 EDA | Corpus coverage, chunk quality, gaps |
| 3 Modeling | Embedding model, chunk strategy, hybrid retriever |
| 4 Training | (Optional) fine-tune embeddings/re-ranker on the domain |
| 5 Evaluation | Retrieval recall, answer faithfulness to source, hallucination rate |
| 5.5 Acceptance | Grounding tests, data leakage, access control |
| 6 Production | Search + generation with low latency; cache; index updating |
| 7 Monitoring | Retrieval quality, hallucination, corpus drift |
| 8 Retraining | Re-index new content; readjust embeddings |
| 9 Governance | Per-document access control, privacy, provenance/citation |
6. Capabilities, modes, and modalities
Intellectual/factual: Q&A over documents, enterprise assistants, conversational search, support. Reduces hallucination and provides traceability (sources). Extends to multimodal RAG (image/audio).
7. Limits, risks, and ethics
Quality limited by retrieval ("garbage retrieval, garbage answer"); poor chunking breaks context; leakage of sensitive data; it can still hallucinate if poorly grounded. Access control is critical.
8. State of the art and examples
Enterprise knowledge assistants, GraphRAG, agentic RAG (iterative retrieval guided by an agent, ch. 30); convergence with long context windows (where "putting everything in context" competes with retrieving).