Glossary · RAG (retrieval-augmented generation)
What is RAG (retrieval-augmented generation)?
Definition: RAG is a technique in which a language model first retrieves information from a company's own documents and then answers from it, citing the source.
The system splits the company's documentation into passages and indexes them, usually as numerical representations of their meaning, known as embeddings, which enable semantic search. When a question arrives, it retrieves the most relevant passages and hands them to the model along with the question. The model writes the answer using that material, so it can show which document each piece of information comes from.
It is the usual way for an assistant or agent to answer with the company's internal knowledge: policies, procedures, contracts, technical data sheets or past incident records. Its practical advantage is that information is updated by changing the documents, without retraining the model. Access can also be filtered so that each user only receives answers based on what they are allowed to see.
RAG reduces hallucinations but does not eliminate them. The quality of the answer depends on the quality of the documents: if they are out of date or contradict each other, the system will reflect that. It also depends on how they are split and indexed. It differs from fine-tuning, which adjusts the model itself with examples and is better suited to teaching a style or format than to supplying changing facts.
Example: an employee asks how many days of leave they are entitled to when moving house, and the assistant answers from the current HR policy, linking to the exact section.