RAG (Retrieval-Augmented Generation)

Retrieval-Augmented Generation (RAG) is an architectural pattern that enhances large language model responses by retrieving relevant information from external knowledge sources and including it in the model's context before generation. This approach addresses fundamental limitations of standalone LLMs: they cannot access information after their training cutoff, they may hallucinate facts, and they lack awareness of private or specialized data.

The RAG pattern was formalized in a 2020 paper by Lewis et al. at Facebook AI Research, but the underlying concept -- retrieving relevant information to inform text generation -- has roots in earlier information retrieval research. The approach gained massive practical adoption with the rise of production LLM applications, as developers quickly discovered that even the most capable models need access to specific, current, and authoritative information to be useful in real-world applications.

A standard RAG pipeline operates in two phases. During the ingestion phase, source documents are processed: they are split into manageable chunks, each chunk is converted into a vector embedding using an embedding model, and these embeddings are stored in a vector database alongside the original text and metadata. During the query phase, the user's question is embedded using the same model, a similarity search identifies the most relevant document chunks, and these chunks are injected into the LLM's prompt as context for generating the response.

The simplicity of basic RAG is both its strength and its limitation. Advanced RAG techniques address quality challenges at every stage of the pipeline. For ingestion, semantic chunking (splitting at natural boundaries rather than fixed sizes), contextual embeddings (enriching chunks with surrounding context), and multi-representation indexing (storing summaries alongside full text) improve the quality of the knowledge base. For retrieval, hybrid search (combining vector similarity with keyword matching), query transformation (rewriting or decomposing queries for better retrieval), and re-ranking (using cross-encoder models to reorder results) improve the relevance of retrieved information.

RAG has become the default architecture for enterprise AI applications because it provides several critical advantages: responses are grounded in specific sources that can be cited, knowledge can be updated by modifying the document store without retraining the model, access control can be enforced at the document level, and the system can work with proprietary data that was never in the model's training set. For most applications, RAG provides a better cost-quality trade-off than fine-tuning for incorporating domain knowledge.

The RAG ecosystem encompasses tools and services across the entire pipeline. Vector databases (Pinecone, Weaviate, Qdrant, Chroma, Milvus, pgvector) provide the retrieval infrastructure. Embedding models from OpenAI, Cohere, Google, and open-source options like BGE and E5 power the semantic representation. Frameworks like LangChain, LlamaIndex, and Haystack provide end-to-end RAG pipeline orchestration.

Managed RAG services reduce the operational burden. Options include OpenAI's Assistants API with file search, Anthropic's contextual retrieval features, Google's Vertex AI Search, and various startup offerings that provide RAG-as-a-service with document processing and retrieval built in.

Evaluation tools like RAGAS, DeepEval, and custom evaluation frameworks help measure RAG quality across dimensions like faithfulness (is the response supported by context), relevance (is the retrieved context relevant to the query), and completeness (does the response cover all relevant information from the context).