What is RAG? Retrieval-augmented generation explained without the jargon

The problem RAG solves

A language model on its own knows what it learned during training. It does not know your price list, your warranty terms, your SOPs or the email thread where a customer agreed to different payment terms. Asked anyway, a model will produce something plausible — which for a business is worse than producing nothing, because a confident invented answer is the one that reaches a customer.

RAG closes that gap without retraining anything. Your documents stay documents. When a question arrives, the system retrieves the handful of passages most likely to contain the answer and hands them to the model along with an instruction to answer only from what it was given, and to say so when the answer is not there.

How a RAG system actually works

1. Ingestion. Your PDFs, DOCX files, spreadsheets, web pages and ticket history are collected and cleaned — including the scanned documents that need OCR before anything else can happen.
2. Chunking. Each document is split into passages small enough to be precise and large enough to keep meaning. Getting this wrong is the single most common reason a RAG system gives vague answers.
3. Embedding. Every passage is converted into a vector — a numerical representation of meaning — and stored in a vector database.
4. Retrieval. A question is embedded the same way, and the passages closest in meaning are pulled back. Good systems combine this with keyword search, because exact terms like part numbers do not embed well.
5. Re-ranking. The retrieved candidates are re-scored by a model that reads them properly, so the best three passages reach the answer step rather than the first three the search returned.
6. Generation. The model writes an answer from those passages and cites the source of each claim.

RAG or fine-tuning?

Fine-tuning teaches a model a style, a format or a specialised task. It does not reliably teach it facts, and every time a fact changes you would have to train again. RAG handles facts: update the document and the next answer is already current.

What good looks like

• Every answer cites the document, and ideally the page or clause, it came from.
• The system says 'I do not have that' rather than guessing when retrieval comes back empty.
• Role-based access, so a public-facing assistant cannot retrieve internal margin data.
• An evaluation set — real questions with known correct answers — scored on every change, so accuracy is a number rather than an impression.
• A path to correct a bad answer: fix the source document, not the prompt.

How much content do you need?

The threshold is repetition, not volume. A few hundred pages that people search every day repay a RAG build faster than a ten-thousand-page archive nobody opens. The question to ask is not 'do we have enough documents' but 'how many hours a week does someone spend finding an answer that already exists in writing'.

A production RAG system — ingestion, retrieval, evaluation and one interface — starts around $999, with model and hosting costs from roughly $29 a month at typical volume.