Building an AI assistant on your own documents

Large language models know a lot about the world and nothing about your contracts, product manuals or internal policies. Retrieval-augmented generation, usually shortened to RAG, closes that gap: before answering, the system searches your documents for relevant passages and gives them to the model as the basis for its answer.

Done well, this gives people answers in seconds with a link to the source. Done badly, it gives confident answers that are subtly wrong. The difference is rarely the model. It is almost always the data and the retrieval.

Your documents are the product

Scanned PDFs, duplicated versions, outdated policies and tables flattened into text all degrade answers. Before any AI work, it is worth deciding which sources are authoritative, removing what is obsolete, and extracting text and tables properly. This step is unglamorous and usually has the biggest effect on quality.

Retrieval decides the answer

If the right passage is not retrieved, the model cannot use it. Good retrieval usually combines several techniques:

• Splitting documents into chunks that keep meaning intact, such as whole sections rather than fixed character counts.
• Combining semantic search with keyword search, so exact terms like product codes are not missed.
• Filtering by metadata such as department, date or document type.
• Re-ranking the top results before they reach the model.

Show the source, every time

Answers should cite the passages they rely on. This lets people check, builds trust, and makes errors visible instead of hidden. It also gives you a way to measure quality: did the answer use the right source?

Respect permissions

An assistant must not become a way to read documents someone is not allowed to see. Access rules from your existing systems should apply to retrieval, so each person only gets answers from what they could already open.

Measure before launch

Collect real questions from the people who will use the assistant, with the answers an expert would give, and test against them. This tells you whether the system is ready and gives you a baseline for every improvement after.

We build assistants like this on company documents and connect them to the tools teams already use. If you are weighing whether it fits your case, we are happy to look at your material and say honestly what to expect.