How to build an AI chatbot trained on your own documents

The hard part of building a useful AI chatbot is not calling a language model — it is making sure the answers come from your material and that the bot admits when it does not know. This guide covers the decisions that actually determine whether people trust it.

1. 01

   Decide what the chatbot must know

   Start from the questions people already ask you. Pull together the documents that answer them: help center articles, product PDFs, pricing pages, returns and warranty policies, onboarding guides. Quality matters far more than volume here — one accurate, current pricing page beats fifty outdated marketing decks, because a chatbot grounded in stale documents will confidently repeat stale information.

2. 02

   Choose a language model

   For most support and documentation chatbots, a small fast model is the right default: GPT-5.4 mini, Claude Haiku 4.5, or Gemini 2.5 Flash all handle grounded question answering well at a fraction of the cost of frontier models. Reserve larger models (Claude Opus, GPT-4.1, o3) for cases needing multi-step reasoning over the retrieved material rather than direct extraction. Because retrieval does the heavy lifting in a well-built system, model size matters less than people expect.

3. 03

   Index your content as searchable passages

   Long documents cannot be sent to a model whole — they exceed context limits and dilute relevance. Content is split into passages (chunks) of roughly a few hundred words, ideally at paragraph boundaries so ideas stay intact. Each passage is converted into an embedding, a numeric vector representing its meaning, which makes semantic search possible: a visitor asking about 'refunds' finds a passage about 'money back' even with no shared words.

4. 04

   Combine keyword and semantic search

   Semantic search alone struggles with exact identifiers — product SKUs, version numbers, proper nouns — because embeddings capture general meaning, not precise strings. Keyword search alone misses paraphrased questions. Running both and merging their rankings (a technique called Reciprocal Rank Fusion) covers each method's blind spot. This hybrid approach is the single biggest lever on answer quality, ahead of model choice.

5. 05

   Ground the model and forbid guessing

   Supply only the retrieved passages to the model, with explicit instructions to answer solely from that evidence and to state plainly when the evidence is insufficient. Treat retrieved content as untrusted data rather than instructions, so text hidden inside an uploaded document cannot hijack the assistant's behaviour. Add a confidence threshold: if retrieval surfaces nothing strong enough, do not call the model at all — say the information is unavailable and offer a human handoff.

6. 06

   Add citations so answers are verifiable

   Show which passages an answer came from. Crucially, attach citations in your application code from the retrieval results, not by asking the model to cite its sources — a model asked to produce citations can fabricate plausible-looking references to documents it never saw. Application-attached citations are verifiable by construction.

7. 07

   Shape tone with examples, not adjectives

   Instructions like 'be friendly and concise' are ambiguous. Two or three example question-and-answer pairs demonstrating the exact voice, length, and formatting you want are far more effective, and they take effect instantly with no model retraining. Keep the example set small: each one consumes tokens on every single response.

8. 08

   Publish, then watch real conversations

   Embed the widget on your site, share a public link, or connect it to a channel like Telegram. Then read the transcripts. The questions people actually ask will differ from what you predicted, and every escalated conversation is a precise signal that a document is missing or unclear. Feeding those gaps back into your knowledge base is what turns a demo into something genuinely useful.