What Is a Knowledge Base Chatbot? How It Works and What It Can't Do

You have probably met the phrase on a pricing page: a chatbot "trained on your data" or "powered by your knowledge base". Both describe the same kind of product, and knowing how it works tells you what it will get right, what it will get wrong and how much upkeep it needs.

This page is for business owners and support leads choosing or running their first AI support agent. It defines the term, walks through the five steps between a question and an answer, then goes deeper on the three things buyers ask about most: what to put in the knowledge base, how it stays current, and whether the bot learns from conversations. The retrieval mechanics themselves are covered in our RAG explainer for non-engineers.

PepoChat is our product; its behaviour below was checked in its code and documentation on 22 October 2026, and every external source was read the same day.

What is a knowledge base chatbot?

A knowledge base chatbot is an AI chatbot that answers visitors' questions from a defined collection of your own content, and admits when that collection has no answer, instead of drawing on the open internet or a model's general training.

The knowledge base is the private, searchable store of text the bot reads from: crawled web pages, uploaded files, synced documents and short answers you write yourself. The phrase has a second meaning, a public help center that customers browse, and many knowledge base chatbots sit on top of one. You do not need help center software to start, though: your website and a few documents already make a knowledge base.

Vendors also call this a chatbot "trained on your data", a document chatbot or a RAG chatbot. The word "trained" is loose, because in almost every product nothing is trained. Your content is indexed so it can be searched, and a language model reads the matching passages at the moment it answers. The technique is retrieval-augmented generation (RAG), which AWS's RAG explainer describes as pointing a model at an authoritative knowledge base outside its training data before it responds, "all without the need to retrain the model".

The consequence for you is practical. The bot is only as complete, current and consistent as its knowledge base, so most wrong answers are content problems you can fix in minutes, not model problems you wait for a vendor to solve.

How is a knowledge base chatbot different from ChatGPT or a rule-based bot?

A knowledge base chatbot differs from ChatGPT in where its answers come from, and from a rule-based bot in whether it can handle questions nobody scripted: it writes its own replies like ChatGPT, but only from your content.

Comparison as of October 2026.

ChatGPT is a general assistant: it knows a great deal about the world, but nothing holds its answers to your current policies, and it lives in OpenAI's apps rather than on your site. A knowledge base chatbot inverts that trade. It knows only what you gave it, so it can say "I don't have that information" instead of producing a plausible guess, and many are built on the same OpenAI models.

A rule-based chatbot follows a decision tree of buttons and keyword triggers; it never writes a sentence nobody scripted, which makes it predictable and brittle. An AI support agent is a knowledge base chatbot with tools and a human handoff added, so it can also check an order, open a ticket or pass the conversation to a person. Most products sold as AI support agents, PepoChat included, are knowledge base chatbots at their core, and the AI agents inside helpdesk suites are the same idea attached to a ticketing system. The definition of an AI support agent covers the tool layer.

How does a knowledge base chatbot work? The five steps

A knowledge base chatbot works in five steps: it ingests your content, splits it into passages, turns each passage into an embedding, retrieves the passages closest to each question, and generates a reply from those passages alone.

Last verified 22 October 2026 in PepoChat's search and ingestion code and its documentation.

An embedding is a list of numbers that represents what a passage means, so a question about "getting my money back" lands near a passage titled "Refund policy" although the words barely overlap. Matching by meaning is what lets the bot handle questions phrased in ways you never wrote.

Two details decide more than the rest. Retrieval runs per question, not per conversation: PepoChat's agent is told to search before answering any product question, and only greetings skip the search, so a follow-up on a new topic gets fresh passages. PepoChat also sets no minimum similarity score. The nearest passages always come back, and the model judges whether they answer the question; when they do not, the agent says it has no specific information in its knowledge base and offers to bring in a person. Agent instructions can change its tone, but not that rule against inventing facts.

What goes into a knowledge base chatbot?

A knowledge base chatbot can use any content that can be turned into text: website pages, uploaded documents, synced Notion or Google Drive pages, help center articles and answers you write yourself, and as of October 2026 PepoChat accepts all five.

Limits from PepoChat's documentation and code, checked 22 October 2026. Free allows 5 knowledge sources and 20 MB of text; Starter and Growth allow unlimited sources with 200 MB and 1 GB.

Start with the pages behind your most common questions: pricing, shipping and returns, account and billing help, opening hours and contact details. Then fill the gaps with documents and Q&A pairs. More is not better. Every irrelevant or outdated passage is one more candidate for the eight places retrieval fills, so a small, accurate knowledge base beats a large, messy one. The guide to formatting docs for an AI chatbot shows how to write each page so its passages still make sense when read alone.

Keep out anything you would not say to every visitor. Unless a product offers per-visitor permissions on its content, and PepoChat does not, any passage can be quoted to anyone who asks the right question. That rules out internal notes, staff-only procedures, unreleased prices, discount codes, customer records and chat transcripts. Transcripts are the tempting one, but they hold names, emails and one-off answers, so read them for gaps and write each fix as a general answer instead.

OWASP's Top 10 for LLM applications, in its 2025 edition, names related risks for retrieval systems: leaks between users who share one store, and data poisoned by insiders or "unverified data providers". Its advice is to accept data "only from trusted and verified sources". For a support bot, that means your own pages and documents, not forums, scraped pages or anything visitors can edit.

Does a knowledge base chatbot learn from conversations?

A knowledge base chatbot does not learn from conversations the way most people assume: the AI model is not retrained on your chats, the knowledge base changes only when someone edits it, and the bot remembers a visitor only within one conversation.

People use "learning" for four different things, and only one of them happens without your involvement.

Checked in PepoChat's code and documentation on 22 October 2026.

Is the AI model trained on my customers' chats?

Most knowledge base chatbots call a model provider's API at answer time rather than training a model, and PepoChat trains and fine-tunes nothing. For each reply it sends the conversation, the retrieved passages and your agent instructions to OpenAI's models under standard API terms. OpenAI's data controls page says that as of 1 March 2023, data sent to the API "is not used to train or improve OpenAI models" unless the customer explicitly opts in, and that abuse-monitoring logs are kept for up to 30 days by default.

Not trained does not mean not stored. Each conversation also sits in the vendor's own database. In PepoChat it stays in the team inbox until you remove it, or until Data retention on the Team page, off by default, deletes resolved conversations after 30, 90, 180 or 365 days. Ask every vendor both questions, training and retention; our guide to where chatbot data goes covers the details.

Does the knowledge base update itself from conversations?

No, and a bot that rewrote its own knowledge from chats would be a liability. In PepoChat, content enters the knowledge base only as imported pages, uploaded files, synced Notion or Drive pages, snippets and Q&A pairs, or published Help Center articles. Nothing a visitor types, and nothing an operator replies in the inbox, is added.

Microsoft showed the risk in 2016. It launched its chatbot Tay expecting it to "get better and better" through interaction, and in the first 24 hours "a coordinated attack by a subset of people exploited a vulnerability in Tay". Microsoft's post-mortem drew the lesson that "AI systems feed off of both positive and negative interactions with people". A support bot that absorbed what visitors typed would invite the same attack, and could repeat one customer's details in another customer's answer.

Does a knowledge base chatbot remember returning visitors?

Within one conversation, yes: the agent reads the earlier messages, so "and on the Growth plan?" makes sense after a question about Free. Across conversations, no. Each new PepoChat conversation starts with no memory of earlier ones; a visitor can scroll back through their own chats in the widget while their 24-hour session lasts, but the agent does not read them.

The exception is meeting memory, an optional PepoChat feature, off by default, that lets the agent recall a returning customer's past meetings from PepoSmart meeting notes: dates, interests, commitments and summaries, never raw transcripts. It works only for a verified visitor, one who proved their email address with a 6-digit code, so a stranger who types someone else's address gets nothing. The meeting memory explainer covers that gate in detail.

What actually improves the answers?

People improve the answers, using what conversations reveal. A weekly routine looks like this:

1. Filter the team inbox by Escalated and read the questions the agent could not answer.
2. Check the Topics card in Analytics for subjects that grow week over week, and the share of thumbs-down votes on the Sentiments card.
3. Name the cause of each miss: missing content, a stale page, two sources that disagree, or a question that needs a live lookup.
4. Fix the source: a Q&A pair worded the way the visitor asked, an edited page followed by Sync website pages, or a replaced file.
5. Ask the same question in the Test the agent drawer, where replies are free and kept out of analytics.

That loop is the honest version of "it learns over time". The learning is yours, and every change is one you can read, date and undo.

How do you keep a knowledge base chatbot current?

You keep a knowledge base chatbot current by changing the source rather than the bot: edit the page or file, then let the weekly re-fetch collect it, or trigger a sync when the change cannot wait.

Checked in PepoChat's documentation and code on 22 October 2026.

A worked example shows why the order matters. Say your returns window drops from 30 days to 14 on 1 November. You edit the returns page, but an old returns PDF and a Q&A pair written in the spring still say 30. Until the next re-fetch, every stored passage says 30; after it, retrieval can return both versions, and the agent may quote either. The fix takes a few minutes: sync the page, replace the PDF, edit the Q&A pair, then ask "How long do I have to return an order?" in the test drawer.

Four habits keep the knowledge base honest between changes:

• Date time-bound facts. "As of November 2026, returns are accepted within 14 days" lets both the agent and the reader judge freshness; "currently" tells neither anything.
• Exclude retired versions. If your docs keep /v1/ pages next to /v2/, add an exclude path to the crawl or sitemap import so old instructions never enter.
• Watch for duplicates. One page under two addresses, such as with and without a query string, is stored twice and can take two of the eight passage places.
• Expect re-syncs to be cheap. PepoChat's weekly re-fetch re-indexes only pages whose words changed, so redeploying an unchanged site costs nothing.

What can't a knowledge base chatbot do?

A knowledge base chatbot cannot answer what is not written down, see a customer's account, act in your systems or make judgement calls, so each of those gaps needs an integration, a person or a rule you set.

• It cannot see account data. "Where is my order?" needs a live lookup, not a page. As of October 2026, PepoChat offers 22 prebuilt actions across 13 providers, such as Shopify order status by order number and Stripe subscription checks, and none of them moves money.
• It cannot act without a tool. Booking a meeting, cancelling a plan or opening a ticket each needs an action you switch on. Action execution is at-least-once, so a write can occasionally run twice, which is why refunds and payments should never be wired as actions.
• It cannot make exceptions. A refund outside policy or a goodwill discount is a judgement call. PepoChat offers a person when nothing relevant is found and escalates to the team inbox when the visitor accepts or asks for one; that human handoff is the hand-over of a conversation, transcript included, to your team.
• It cannot guarantee a correct answer. Grounding narrows mistakes without ending them, because the model can misread a passage or faithfully quote a stale one. Test with questions it should refuse, as the guide to stopping chatbot hallucinations explains.
• It cannot keep a secret that sits in its knowledge base. Anything stored can be quoted to any visitor.

PepoChat has three limits of its own worth knowing, as of October 2026. Replies do not show which page they came from, in the widget or in the team inbox. Search matches by meaning only, with no separate keyword search, so exact strings such as SKUs or error codes can match less reliably than sentences. And there is no report of unanswered questions, so the Escalated filter and the Topics card are where the gaps show up.

How do you choose a knowledge base chatbot? A 10-point checklist

Choose a knowledge base chatbot by checking ten things before the price: grounding, citations, human handoff, source types, re-sync, permissions, pricing unit, data policy, widget weight and test tools, all of which a free plan or trial can show.

PepoChat column checked in its code, documentation and pricing page on 22 October 2026; loader size measured on the live widget the same day.

Two rows deserve a closer look. On pricing, ask what happens at the cap: when a PepoChat Free workspace reaches 200 replies, new conversations go to the team inbox with a friendly message, while paid plans keep answering and bill $10 per 1,000 extra replies. On grounding, the quickest test is a refusal test: ask five questions your content does not answer and count how many times the bot admits it cannot help instead of guessing.

What to do next

• Build one in an afternoon with the guide to training an AI chatbot on your website and PDFs, starting with the pages behind your top ten questions.
• Trace a single reply from question to handoff in the PepoChat docs page on how answers work.
• Create a free workspace to run the refusal test on your own content: every feature is on the free plan, with 200 AI replies a month and 5 knowledge sources, and no card to enter.