What isRAG?
Retrieval augmented generation connects a language model to a search over documents you decide on.
Get in touch
Let's talk about your project.
First we check whether the project fits your business model. Then you get a proposal with phases and effort.
Talk about a RAG project or call: +49 151 1576 5566RAG (retrieval augmented generation) is a method in which an AI looks things up in documents you have chosen before it answers. A search step pulls out the passages that fit the question and hands them to the language model along with it, so the reply rests on your own material and can be checked against its source.
- The model itself stays unchanged and only gets relevant reading material with each question, so updated documents take effect as soon as they are re-indexed.
- Invented claims become rarer, but they do not disappear.
How does RAG get your company knowledge into an AI's answers?
A large language model only knows what was in its training data. Last quarter's price list was certainly not part of it, so the model may piece together an answer that sounds right and is wrong. With RAG, a program first searches your service manuals, contracts or help articles, and the model then writes from what is actually there. People often ask for an AI assistant trained on company data. RAG usually fits that wish better than retraining, because the model reads the relevant passages at the moment of the question. It pays off most where many people keep searching the same folders.
How does RAG find the right passage in your documents?
Documents are cut into chunks of a few paragraphs, and each chunk is turned into a string of numbers that roughly captures its meaning. Those numbers go into a vector database. A question gets the same treatment, and the database returns the closest chunks (a part number is often easier to find by keyword, so many systems combine both). Most of the effort sits in this search, not in the model: if the 2024 price list sits next to the 2026 one, the answer may rest on the wrong version.
Where you should still check the source when you use RAG
RAG makes hallucinations rarer without ending them. In a 2024 Stanford study, legal research tools whose vendors had promoted RAG beat GPT-4 without search. Still, depending on the tool, they got between a sixth and a third of the test queries wrong or misleading. Where contracts, health or money are involved, a person should read the source before acting. And since a RAG system searches everything you give it, salary lists do not belong in the same index as the product manual unless the search checks permissions.
Frequently asked questions
What is the difference between an LLM and RAG?
The LLM is the language model that writes. RAG is the setup around it that hands it relevant passages from a search before every answer.
Is RAG compatible with the GDPR?
It depends on the setup. With a cloud provider you will often need a data processing agreement, and if you want to be on the safe side, you run it on your own servers.
Terms you should know in the same context
Vector database · Fine-tuning · AI hallucination · LLM (large language model) · AI assistant · Back to the AI glossary A to Z
Whether RAG holds up day to day depends less on the model than on your documents and on who is allowed to see them.