Part 06 — Retrieval-Augmented Generation

How it is grounded in facts.

A language model’s knowledge is frozen at its training cut-off, is unverifiable, and cannot be corrected without retraining. Retrieval solves the first and third of those problems and gives you the citation needed for the second. RAG is therefore less “a technique” than a pipeline whose quality is the product of several independent stages’ quality.

The chapters follow the data through that pipeline.

1. The RAG Pipeline — the end-to-end path from query to cited answer, the naive baseline, and how each stage contributes to failure.
2. Chunking — why fixed-size splitting destroys meaning, the strategies that preserve it (recursive, semantic, late chunking), and how chunk size trades recall against context cost.
3. Embedding Models — what a vector actually represents, bi-encoders vs cross-encoders, dimension and Matryoshka truncation, and how to choose a model with MTEB rather than vibes.
4. Retrieval: Hybrid Search and Reranking — why BM25 still beats dense retrieval on rare tokens, how reciprocal rank fusion combines them, and where a cross-encoder reranker earns its latency.
5. Query Transformation — rewriting, HyDE, decomposition, and multi-query expansion: fixing retrieval by fixing the query instead of the index.
6. RAG Evaluation — retrieval metrics with exact formulas (recall@k, MRR, nDCG), generation metrics (faithfulness, answer relevance), the biases of LLM-as-a-judge, and how to build a regression set.

The honest framing: most RAG systems that disappoint are not failing at generation. They are retrieving the wrong chunks, and nobody measured it.

Depends on: Part 01 for the bidirectional encoder representations the embedder is built from.

Where this leads: Part 07 puts retrieval behind a router inside a shipped system.