TL;DR
Retrieval-Augmented Generation (RAG) retrieves relevant passages from an external knowledge base before generating an answer. It is the mainstream architecture for grounding LLMs with fresh, private, or verifiable information.
Key Points
- Why: Model parameters have a knowledge cutoff and cannot cover private data. RAG externalizes memory into a database.
- Three stages:
- 1. Ingestion: split documents into chunks → embed → store in a vector database;
- 2. Retrieval: embed the query, do similarity search (optionally hybrid keyword+vector, plus reranking);
- 3. Generation: place retrieved passages into the prompt and generate with citations.
- Common variants: HyDE (hypothetical document embeddings), GraphRAG (graph-based), parent-document retrieval (small chunks retrieved, larger chunks fed).
- Key knobs: chunk size/overlap, embedding model, top-k, similarity threshold, reranker.
Trade-offs
- Pros: controllable, updatable, traceable, fewer hallucinations.
- Cons: added retrieval latency, infrastructure cost, answer quality bounded by retrieval quality.
- Not a silver bullet: multi-hop reasoning and global summarization may need long context or graph methods.
Sources
- Lewis et al., "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks" (2020)
- Wikipedia: Retrieval-augmented generation
- Official docs: Pinecone / LangChain / LlamaIndex