Home / Learn / What is retrieval-augmented generation (RAG)?
Deepskilling Learn
What is retrieval-augmented generation (RAG)?
RAG separates “what the model knows” from “what your systems store.” It is the default pattern for enterprise Q&A, copilots, and policy-aware assistants when citations and freshness matter.
DEFINITION
Retrieval-augmented generation (RAG) is an architecture that retrieves relevant documents at query time and conditions a language model on that context, so answers stay grounded in up-to-date or proprietary knowledge without full model retraining.
How RAG works
A query is embedded and matched against a vector (or hybrid) index. Top passages are assembled into a prompt with instructions and guardrails. The LLM generates an answer that should cite or stay within retrieved evidence. Evaluation checks faithfulness, relevance, and latency—not just fluency.
When RAG is the right default
Choose RAG when knowledge changes often, legal/compliance needs citations, or you must keep secrets out of fine-tuned weights. Weak retrieval, poor chunking, or missing eval harnesses cause most “RAG failures,” not the base model alone.
Production checklist
Define document ownership and refresh SLAs; choose chunking and metadata filters; measure recall@k and groundedness; add refusal behaviour for empty retrieval; log prompts/retrieval for incident review; budget p95 latency and cost per answer.
FAQ
Is RAG the same as fine-tuning?
No. RAG injects external context at inference; fine-tuning changes model behaviour via training. They solve different problems and are often combined.
Do I need a vector database for RAG?
Usually yes for semantic search at scale, though hybrid BM25 + embeddings, or managed stores in Bedrock/Azure AI Search, are common. The index quality matters more than the brand of store.
Expert
Rajeev Chandran — Founder & Curriculum Architect, Deepskilling. Designs practice-first AI, cloud, and systems programmes—from RAG and GPU labs to executive GenAI decision frameworks.
Case studies
Research citations
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Lewis, P. et al. (2020). NeurIPS. Foundational RAG formulation pairing parametric models with non-parametric memory.