What is RAG — and when does your business actually need it?

Large language models are remarkable writers and terrible librarians. Ask one about your leave policy, your product's warranty terms or last quarter's pricing and it will produce a confident, fluent — and frequently wrong — answer. Retrieval-augmented generation (RAG) fixes that by giving the model the right pages to read before it answers.

How RAG works, in plain terms

A RAG system has two halves. The indexing half reads your documents, splits them into passages, and stores a numeric "meaning fingerprint" (an embedding) for each passage in a vector database such as PostgreSQL with pgvector. The answering half takes a user's question, finds the passages whose meaning is closest, and hands them to the model with one instruction: answer only from these, and cite them.

1. Ingest: PDFs, Word files, spreadsheets, web pages and database rows are cleaned and chunked.
2. Embed: each chunk becomes a vector that captures what it is about.
3. Retrieve: a question is embedded the same way; the nearest chunks are fetched, often combined with keyword search.
4. Generate: the model writes an answer grounded in those chunks, with citations back to the source.

Why not just fine-tune a model?

Fine-tuning teaches a model a style or a narrow skill. It is a poor way to teach it facts that change — prices, policies, inventory, contracts. RAG keeps knowledge in a database you control: update a document and the very next answer reflects it. You also get citations, access control per document, and the ability to switch models without retraining anything.

Five signs your business is ready for RAG

• Your team answers the same questions from the same documents every week.
• Critical knowledge lives in PDFs and shared drives that nobody searches well.
• Customers wait hours for answers that already exist in your manuals.
• New hires take months to become productive because "it's all somewhere".
• You need answers to respect permissions — not everyone should see everything.

What makes a RAG system production-grade

A weekend prototype can impress in a demo and fail on day three. The difference is engineering discipline: chunking tuned to your document types, hybrid retrieval, permission filters applied at query time, refusal when sources are missing, and — most importantly — an evaluation set of real questions with known good answers, re-run every time a prompt or model changes.

Where to start

Pick one high-volume, well-documented workflow — HR policy questions, product support, or internal SOPs. Collect fifty real questions. Build, measure, then expand. That is exactly how we run our RAG engagements, and why our assistants stay accurate after launch.