What is RAG?.
RAG (retrieval-augmented generation) is a technique that lets an AI model answer from your documents. Before writing, the system searches your knowledge base for the relevant passages and hands them to the model, which uses them to answer, citing its sources.
The problem it solves is simple. An LLM such as Claude, GPT or Mistral was trained on huge volumes of public text, up to a given date. It does not know your procedures, prices or contracts. Asked about your business, it makes things up or admits it does not know.
RAG gives it, for each question, the right excerpts from your documents. It is like asking a very capable colleague to answer, after first handing them the three pages of the binder that cover the topic.
How RAG works.
A RAG system always runs four steps:
| Step | What happens | Example |
|---|---|---|
| 1. Prepare | Documents are split into passages and indexed so they can be found by meaning, not just by keywords | The employee handbook, split section by section |
| 2. Retrieve | For each question, the system finds the passages closest in meaning to the question | "How many remote days?" brings back the remote work section |
| 3. Augment | The retrieved passages are added to the instructions sent to the model | "Answer only from these excerpts" |
| 4. Generate | The model writes the answer and cites its sources | "Two days a week, according to section 12" |
Step 1 happens once, then at each document update. Steps 2 to 4 happen for every question, in seconds.
RAG or fine-tuning: which to choose?.
There are two main ways to give an AI business-specific knowledge:
| RAG | Fine-tuning | |
|---|---|---|
| Principle | Provide the right documents at question time | Retrain the model on your data |
| Updates | Update the knowledge base | Retrain the model |
| Cites sources | Yes | No |
| Switching models | Easy, the base stays | Everything must be redone |
| Typical case | Changing knowledge: procedures, offers, documentation | Very specific style or format, very narrow repetitive task |
For business knowledge, which changes all the time, RAG is almost always the right starting point. It has another advantage: it ties you to no model. The day a better model comes out, you plug it into the same base.
What RAG changes for a business.
A few uses where RAG saves time every day:
- The internal assistant. Teams ask about procedures, HR benefits or billing rules, and get a sourced answer instead of interrupting a colleague.
- Customer support. A chatbot that answers from your documentation and terms, and hands over to a person when it finds nothing.
- Pre-sales. Finding the right customer reference, clause or data sheet in seconds for a tender response.
- The AI agent that checks before acting. An agent that reads the refund policy before approving a request, or a contract's terms before following up with a customer.
In that last case, RAG becomes one building block of a larger system: that is what sets an AI agent apart from a plain chatbot.
Pitfalls to avoid.
A RAG system is easy to demo and harder to make reliable. The most common traps:
- Outdated or contradictory documents. If two versions of a procedure coexist, the AI may cite the wrong one. RAG is only as good as its base: cleaning it up first is essential.
- Poor splitting. Passages that are too short lose context, too long drown the information. Splitting is tuned to each document type.
- No "I don't know" answer. When the search finds nothing relevant, the system should say so rather than let the model improvise.
- Ignored access rights. An employee must not get, through the assistant, a document they are not allowed to read. Permissions are handled from the design stage.
- No measurement. Without a reference set of questions and answers, there is no way to know whether a change improves or degrades the system.
To go further: understand the underlying building block with What Is an LLM, and see how an agent combines RAG, tools and guardrails in our custom AI agents.
RAG: what people ask us.
What is RAG in AI?
RAG (retrieval-augmented generation) is a technique that lets a language model answer from your documents. Before writing, the system searches a knowledge base for the relevant passages and hands them to the model, which uses them to answer while citing its sources.
What is the difference between RAG and fine-tuning?
Fine-tuning changes the model itself by retraining it on your data: slow, costly, and to be redone at every change. RAG leaves the model untouched and gives it the right documents at question time: you update the knowledge base, not the model. For business knowledge that keeps changing, RAG is almost always the right choice.
What is RAG used for in business?
To answer reliably from your own content: an internal assistant that knows your procedures, a customer chatbot grounded in your documentation, an agent that checks a contract or a policy before acting. The common thread: answers specific to your business, with sources.
Does RAG eliminate hallucinations?
It greatly reduces them, without eliminating them. Giving the model the right passages and asking it to stick to them limits made-up answers. But if the search returns the wrong document, or none, the model can still go wrong. That is why citing sources and allowing an "I don't know" answer matter.
What documents can go into a RAG system?
Any text content or anything convertible to text: PDFs, web pages, Word documents, product sheets, support tickets, knowledge bases, emails, transcripts. Documents should be up to date and cleanly split so the search finds the right passage.
What is the difference between RAG and MCP?
RAG gives the AI knowledge (documents to read). MCP gives the AI access (tools to use, like a CRM or a database). Both often work together in an agent: RAG to know, MCP to act. See What Is MCP in AI.