RAG in practice: how to connect your company knowledge base to AI
RAG (Retrieval-Augmented Generation) is how you get AI to answer from your documents instead of from the internet. The model first retrieves the right passages from your company knowledge base, and only then writes an answer — with a pointer to where it came from. Below we show what such an implementation looks like in practice: from choosing sources, through permissions, to the pilot.
- RAG answers questions strictly from your own documents and always points to the source of the answer.
- The bulk of the work is order in the sources and permissions, not the model itself — that is where quality is decided.
- A pilot on one area of knowledge, for example support procedures, runs 2–3 weeks after the audit.
What RAG is and when you need it
The typical sign that a company needs RAG is a queue of questions for "the person who knows": what is the returns policy for B2B, which price list applies to this customer, what does the service contract say. The answers exist — in PDFs, emails, and the CRM — but finding them takes minutes or means pulling someone away from their work.
A RAG system indexes those sources and answers in seconds. The key difference from an ordinary AI chat: the model does not guess. If the answer is not in the documents, it says so plainly, and it backs every answer with a specific source — a file, a page, a CRM record.
How long it takes to find an answer in your procedures: manually, leafing through documents and asking colleagues — and with a RAG system connected to three knowledge sources.
Which sources to connect first
Not all of them at once. The best first sources are the ones your team asks about most often and that have a single owner: policies and procedures, price lists, framework agreements, product documentation. Case history from the CRM and the inbox comes second, because it takes more work on permissions.
Quality of sources matters more than their number. One current price list works better than five versions scattered across folders. Part of the audit is exactly that decision: which documents are the "source of truth" — and who is responsible for keeping them current.
Permissions and the safety of answers
RAG has to respect who is asking. A sales rep can see the price list and the terms for their own customer, but not the board's margins. An end customer sees public documents only. That is why permissions are set at the level of sources and roles before indexing, not after the fact.
Data stays in your infrastructure or in a chosen EU region, and every query and every answer is logged. So at any moment you know who asked what, and which documents the system based its answer on.
Implementation step by step
Knowledge audit. We pick 1–2 areas, list the sources and their owners, and set permissions. An hour of conversation plus a review of the documents.
Indexing and pilot. We connect the sources, build the retrieval layer, and test it on the team's real questions. The pilot is running in 2–3 weeks.
Quality check. The team rates the answers on a sample of 50–100 questions. We fix indexing wherever the system misses the right source.
Rollout and growth. RAG goes to the team — in Slack, Teams, or on your website. Further sources are added iteratively, under the same permission rules.
From the team's point of view nothing changes in the tools: you ask questions where you already work. The only thing that changes is how long you wait for an answer — and the fact that nobody has to go looking for it by hand.
Frequently asked questions
How is RAG different from an ordinary AI chat?
An ordinary chat answers from the model's general knowledge and may guess. RAG first retrieves passages from your documents, answers only on that basis, and always cites the source. If the answer is not in the knowledge base, it says so plainly.
Does RAG hallucinate?
The risk is many times lower than in an open chat, because the answer has to follow from the cited documents. On top of that, every answer can be verified with one click through to its source — and we measure quality on a sample of real questions before the system goes live.
How long does a RAG implementation take?
We launch a pilot on one area of knowledge 2–3 weeks after the audit. A full implementation with permissions and further sources usually takes 4–8 weeks.