What Is an AI Hallucination? Causes, Examples and How to Prevent It
An AI hallucination is a false or made-up AI answer stated as fact. In Vectara's 2026 summary test, even the best of 111 models did it 1.8% of the time.
Short answer
An AI hallucination is an AI-generated answer that is false or made up but stated as fact: an invented court case, a wrong number, a refund policy that does not exist. Language models produce them because they predict plausible text and are scored in ways that reward guessing. Grounding in sources reduces them; nothing removes them entirely. PepoChat's agent answers from your knowledge base and offers a person instead of guessing, free for 100 AI replies a month.
You asked a chatbot a simple question and got a fluent, specific, confident answer that turned out to be wrong. Maybe it cited a study that does not exist, quoted a price nobody ever charged, or described a feature the product never had. That is an AI hallucination.
This guide is for business owners, support leads and students deciding how far to trust an AI tool. It covers where the term came from, why models hallucinate, the main types, the court cases that made it a legal issue, how often it happens, and how to reduce it and spot it.
What is an AI hallucination?
An AI hallucination is when an AI system, usually a chatbot built on a large language model, states something false or made up as if it were a fact. The answer sounds as confident as a correct one. The model is not lying; it is producing plausible text that happens to be untrue.
| Question | Short answer | Where it comes from |
|---|---|---|
| What is it? | A confident answer that is false or not supported by any source | NIST AI 600-1 (2024); Kalai et al. (2025) |
| Other names | Confabulation, fabrication | NIST AI 600-1 |
| Main cause | Predicting plausible text, plus training and tests that reward guessing | Kalai et al. (2025) |
| Well-known example | Six invented court decisions in a New York court filing | Mata v. Avianca (2023) |
| Not the same as | A lie (no intent), a bias (a systematic skew) or a simple slip | Explained below |
| Can it be fixed? | Reduced with grounding, abstaining and review; not eliminated | Explained below |
The US National Institute of Standards and Technology (NIST) defines confabulation in its Generative AI Profile, NIST AI 600-1 (July 2024) as "the production of confidently stated but erroneous or false content (known colloquially as 'hallucinations' or 'fabrications') by which users may be misled or deceived." It also covers output that contradicts the prompt or the model's own earlier statements. OpenAI researchers, in a 2025 paper discussed below, call hallucinations "plausible falsehoods".
Where does the term "AI hallucination" come from?
The term "hallucination" comes from computer vision, where it first meant something useful: Simon Baker and Takeo Kanade's 2000 paper "Hallucinating Faces" used it for software that invents plausible detail to sharpen low-resolution images of faces. There, making things up was the goal. A 2022 survey by Ziwei Ji and colleagues, Survey of Hallucination in Natural Language Generation, published in ACM Computing Surveys, traces the word back to that paper.
Language researchers then borrowed the word for a failure. From the late 2010s, studies of machine translation and summarization used "hallucination" for output that was not supported by the text the system was given. On 15 November 2023, Cambridge Dictionary named "hallucinate" its Word of the Year and added the meaning "When an artificial intelligence ... hallucinates, it produces false information."
Not everyone likes the word. NIST chose "confabulation" and notes in a footnote that some commenters said "hallucination" and "fabrication" anthropomorphize AI, which is itself a risk because it attributes human traits to software. The OpenAI paper makes the same point more briefly: the error "differs fundamentally from the human perceptual experience".
Why do AI models hallucinate?
AI models hallucinate because a language model generates the most plausible next words, not verified facts, and because the way models are trained and tested rewards a confident guess over "I don't know". Gaps, contradictions and stale pages in the sources a system reads add a third cause. The clearest account of the first two is Why Language Models Hallucinate, a September 2025 paper by Adam Tauman Kalai, Ofir Nachum and Edwin Zhang of OpenAI and Santosh Vempala of Georgia Tech.
How a hallucination happens, step by step
Follow one question through. A shopper asks a general-purpose chatbot, with no access to the shop's website: "What is the return window at Juniper & Pine?" (an invented small shop, used here for illustration).
- The model reads the question as a pattern. It has no database to look up. It predicts which words are most likely to come next, based on everything it saw in training.
- It finds a strong general pattern and no specific fact. It has read thousands of return policies, and "within 30 days" is a common phrase in them. It has never seen Juniper & Pine's policy, or saw it once at most.
- Rare facts are where guessing starts. For arbitrary facts with no learnable pattern, such as birthdays, the paper shows a base model's error rate should be at least the share of facts that appeared only once in training: if 20% of birthday facts appear exactly once, expect at least 20% of birthday answers to be wrong.
- Training taught it that answering pays. Most benchmarks grade answers right or wrong, so "I don't know" scores zero and a guess sometimes scores a point. The authors put it bluntly: language models are "optimized to be good test-takers, and guessing when uncertain improves test performance."
- Out comes a fluent, specific, wrong answer. "Juniper & Pine accepts returns within 30 days of delivery." If the real window is 14 days, the shopper now believes something false, and nothing in the wording warned them.
Ask the same question again and you may get "60 days". The paper reports exactly this with a real model: asked three times for Kalai's birthday, and told to answer only if it knew, DeepSeek-V3 gave three different wrong dates.
Gaps and conflicts in the sources
Systems that read documents before answering, a design called retrieval-augmented generation (RAG), fail in a third way: the right page is missing, two pages disagree, or the page is out of date. The model then fills the gap or picks the wrong version. The paper notes that search and RAG reduce hallucinations but are "not panaceas", because grading still rewards a guess whenever the search comes back empty. Stale pages are one reason PepoChat re-fetches website sources weekly on paid plans and re-indexes only the pages that changed. Our explainer on RAG for customer support shows where that retrieval step breaks.
What are the types of AI hallucinations?
The common types of AI hallucination are made-up facts, made-up citations or links, wrong numbers, invented product features or policies, and confident answers about events after the model's training ended. Each has a real or typical example.
| Type | What happens | Example | How to catch it |
|---|---|---|---|
| Made-up fact | States a specific fact it never learned | Three different wrong birthdays for the same researcher (Kalai et al., 2025) | Ask again; check one primary source |
| Made-up citation or link | Invents a source, quote or URL that looks real | "Varghese v. China Southern Airlines", a case that does not exist (Mata v. Avianca) | Open the source; search the exact title |
| Wrong number or count | Miscounts, miscalculates or misquotes a figure | Asked how many Ds are in DEEPSEEK, DeepSeek-V3 answered 2 or 3 in ten tries; the answer is 1 (Kalai et al.) | Recalculate it yourself |
| Invented policy or feature | Describes a rule or feature the business never had | A refund rule for bereavement fares that the airline did not offer (Moffatt v. Air Canada) | Compare with the official policy page |
| Past the training cutoff | Treats outdated knowledge as current | A model trained before a price change quotes the old price as today's | Ask what date its information runs to |
Researchers also split hallucinations by how they relate to the source text. An intrinsic hallucination is output that contradicts the source the model was given, such as a summary that changes a date in the document. An extrinsic hallucination adds something the source cannot confirm. Ji and colleagues note that an extrinsic addition is sometimes true, which is exactly why it is hard to catch: it reads like useful background.
What are real examples of AI hallucinations?
Well-documented real examples of AI hallucinations include six fake court decisions cited in Mata v. Avianca (2023), an Air Canada chatbot's invented bereavement refund rule (2024), and 18 non-existent citations in an English case (2025). Court records are the best evidence, because judges write down exactly what went wrong.
Mata v. Avianca (United States, 2023): six invented court decisions
Roberto Mata sued the airline Avianca in New York, saying a metal serving cart struck his knee on a flight. His lawyers' filing opposing dismissal cited cases including "Varghese v. China Southern Airlines". Six of the decisions it relied on did not exist. In the Opinion and Order on Sanctions of 22 June 2023, Judge P. Kevin Castel of the US District Court for the Southern District of New York found the fake opinions were "created by the artificial intelligence tool ChatGPT".
The detail that matters most: when the lawyer asked ChatGPT whether Varghese was a real case, it said the case "does indeed exist" and could be found on Westlaw and LexisNexis. Asking the model to check itself produced a second hallucination. The court imposed a $5,000 penalty and ordered letters to each judge falsely named as an author.
Moffatt v. Air Canada (Canada, 2024): a chatbot's invented refund rule
In November 2022, after their grandmother died, Jake Moffatt asked Air Canada's website chatbot about bereavement fares. The chatbot said a reduced fare could be claimed within 90 days of the ticket being issued. Air Canada's actual policy did not allow retroactive claims. In Moffatt v. Air Canada, 2024 BCCRT 149, decided on 14 February 2024, British Columbia's Civil Resolution Tribunal rejected the airline's argument that it was not liable for its chatbot.
Tribunal member Christopher C. Rivers wrote: "It should be obvious to Air Canada that it is responsible for all the information on its website. It makes no difference whether the information comes from a static page or a chatbot." He found negligent misrepresentation and ordered $650.88 in damages, plus interest and fees, $812.02 in total.
Ayinde and Al-Haroun (England, 2025): 18 of 45 citations did not exist
On 6 June 2025 a Divisional Court of the High Court in London gave a joint judgment on two referrals, Ayinde v London Borough of Haringey and Al-Haroun v Qatar National Bank, [2025] EWHC 1383 (Admin). In one case a barrister's grounds cited five cases that did not exist. In the other, 18 of 45 citations put before the court did not exist, and one fake decision was attributed to the very judge hearing the case.
The court's description is one of the plainest any official body has given: freely available tools "can produce apparently coherent and plausible responses to prompts, but those coherent and plausible responses may turn out to be entirely incorrect." The court said lawyers have a professional duty to check AI research against authoritative sources before using it.
How many court cases involve AI hallucinations?
The AI Hallucination Cases database maintained by researcher Damien Charlotin, which tracks decisions where a court or tribunal found (or implied) that a party relied on hallucinated content, plus a few where AI use was only alleged, listed 2,292 cases when checked on 12 November 2026. It does not count filings nobody caught. This is not legal advice; it shows how often the problem reaches a judge.
How often do AI models hallucinate?
How often AI models hallucinate depends on the task and the model, so any single "hallucination rate" is misleading. One transparent public measure is Vectara's hallucination leaderboard, which asks each model to summarize more than 7,700 articles using only facts in the article, at temperature 0 where possible, and uses a separate evaluation model (HHEM-2.3) to flag summaries that are not consistent with the article.
| Summary test measure (leaderboard updated 22 October 2026) | Value | What it tells you |
|---|---|---|
| Models listed | 111 | Wide coverage of current and older models |
| Lowest hallucination rate | 1.8% | Even the best model added unsupported facts |
| Median rate | 9.6% | A typical model erred in about 1 summary in 10 |
| Highest rate | 24.2% | About 1 summary in 4 |
| Models under 5% | 10 | Low rates are the exception |
| Models at 10% or more | 51 | Close to half |
Figures read from the leaderboard on 12 November 2026; they measure summaries of a given article, not hallucination in general. The median and counts are our arithmetic on its table.
Summarizing a document the model has in front of it is close to the easiest case. Open-ended questions about obscure facts give the model far more room to guess, and rates change with each model release, so treat any figure as a dated snapshot.
Is an AI hallucination the same as an error, bias or confabulation?
An AI hallucination is one kind of error: a plausible falsehood stated with confidence. Not every mistake is a hallucination, and the OpenAI paper says so directly ("not all errors are hallucinations").
| Term | What it is | Example | Use this word when |
|---|---|---|---|
| Hallucination | A confident, plausible statement that is false or unsupported | A made-up court case with a real-looking citation | The output invents or misstates facts |
| Error | Any wrong output, including obvious slips | A garbled sentence or a formatting mistake | You mean "wrong" in general |
| Bias | A systematic skew from the training data or design | Consistently different answers depending on a person's name | The problem repeats in one direction across many answers |
| Confabulation | NIST's formal term for hallucination | The same made-up court case | You write policy, risk or compliance documents |
| Lie | A falsehood told on purpose | A person knowingly misquoting a contract | Someone intended to deceive; models have no intent |
How do you prevent AI hallucinations?
You prevent most AI hallucinations by grounding answers in trusted sources, letting the system say "I don't know", checking the output, and keeping a person in the loop for anything consequential. No single technique removes them; each one fixes a different cause.
| Technique | What it fixes | What it does not fix |
|---|---|---|
| Grounding (RAG): answer from your own documents | Guessing about facts the model never learned | Wrong, missing or contradictory documents |
| Restrict answers to the sources | Padding answers with outside "knowledge" | A source that is itself out of date |
| Allow "I don't know" and hand off | Confident guesses when nothing was found | Unhelpful replies if the knowledge base is thin |
| Ask for citations or exact quotes | Unverifiable claims; makes checking fast | Fake citations, unless the system pulls them from real documents |
| Lower temperature | Random variation between answers | Confident errors the model makes every time |
| Ask again or cross-check with a second source | One-off guesses (they change between runs) | Errors the model repeats consistently |
| Human review | Anything the other steps miss, before it reaches a customer | Scale: people cannot read every reply |
Ground answers in sources and let the model abstain
Grounding does the heavy lifting because it changes where the facts come from: from your documents instead of the model's memory. Pair it with an explicit rule that, when the documents do not cover a question, the system says so. The OpenAI paper proposes the same idea for benchmarks: tell the model to "answer only if you are" more confident than a stated threshold, and score a wrong answer below an honest "I don't know". PepoChat's agent, for example, searches the workspace's knowledge base before it answers any product question.
Lower the temperature, but know its limits
Temperature is a setting that controls how random the word choice is; OpenAI's API reference describes lower values as "more focused and deterministic". Lower temperature makes answers more repeatable, which helps testing. It does not make them true: the leaderboard above ran models at temperature 0 where possible and still measured 1.8% to 24.2%.
Check and review the output
For support teams, the full fix list (restricting scope, citing sources, cleaning contradictory pages, treating retrieved text as untrusted) is in our guide to stopping chatbot hallucinations. The short version: test with questions the bot should refuse, then read real transcripts every week.
How can you spot an AI hallucination?
You spot an AI hallucination by checking the claim against a source you open yourself, asking for the exact quote, and asking the same question again to see whether the answer changes. Fluency is not evidence; hallucinations read just as smoothly as correct answers.
Check the source yourself
Open every link and search the exact title of every study, case or article. If you cannot find it in a database or on the publisher's site in two minutes, treat it as invented until proven otherwise.
Ask for the exact quote
Ask: "Quote the sentence from the source that says this." Then find that sentence in the source. A paraphrase can drift; a quote either appears in the document or it does not.
Ask again, in a new chat
A guess is unstable. The birthday example in the OpenAI paper produced three different dates in three tries. If the second answer differs from the first on a factual detail, neither is reliable.
Watch for warning signs
Be most careful with very specific details on obscure topics (names, dates, page numbers, prices), anything that happened recently, arithmetic, and answers that agree a little too readily with the premise of your question.
Can AI hallucinations be eliminated?
AI hallucinations cannot be eliminated from today's language models; they can only be made rarer and easier to catch. NIST calls confabulations "a natural result of the way generative models are designed", because the models approximate the statistical patterns of their training data. In a formal analysis, Ziwei Xu, Sanjay Jain and Mohan Kankanhalli argued in January 2024 that "it is impossible to eliminate hallucination in LLMs" used as general problem solvers.
The OpenAI paper adds a nuance: a program that answered only from a fixed question-and-answer database plus a calculator, and said "I don't know" to everything else, would never hallucinate, though it could answer only that fixed set of questions. Some facts, such as birthdays absent from the training data, cannot be learned at all, so the only way to avoid a hallucination there is to abstain. The authors also note that hallucinations "are still present in the latest models".
In practice that means a trade-off you choose deliberately: a system that answers fewer questions but rarely guesses, backed by a human for the rest, or one that answers everything and is sometimes confidently wrong. For anything a customer acts on, choose the first.
What do AI hallucinations mean for a business?
For a business, an AI hallucination is a statement made by the business. If your website chatbot invents a discount, a refund window or a feature, a customer can rely on it, and the Air Canada decision shows a tribunal may hold you to it. The English court said the same of lawyers: using AI does not move the duty to check. Rules differ by country, and this is not legal advice.
Treat an AI support agent like a new employee who answers from the handbook. Decide what it may answer, give it a current, contradiction-free knowledge base, tell it to say "I don't know" and hand the conversation to a person when the handbook is silent, and keep money, refunds and account changes behind human approval. Our post on AI chatbot security risks covers the wider liability picture.
Then measure before you launch. Run the 40-question chatbot test script, including questions your content does not answer, and read the replies. A bot that says "I don't know" too often is a content problem you can fix with a page; the guide to "I don't know" replies lists the causes. A bot that never says it is the bigger risk. In regulated fields, keep sensitive personal or medical details out of the chat entirely, whichever tool you use.
Where PepoChat fits
PepoChat is an AI support agent for websites, and its design follows the fixes above. For any product or service question, the agent searches your workspace's private knowledge base first (website pages, documents, Q&A pairs, Notion and Google Drive pages, Help Center articles) and writes its reply from what it finds. Its base rules tell it never to invent facts, and they take priority over your own agent instructions, which set its tone and the topics it covers.
When the search finds nothing relevant, the agent says it does not have that information and offers to connect the visitor with a person, instead of guessing. Accepted handoffs, and visitors who ask for a human, land in the team inbox as escalated conversations with the full transcript.
Before launch, agent tests let you save scripted test conversations (up to eight visitor messages each) with a plain-language success criterion, such as "never promises a refund amount"; a grading model scores each run against it.
The limits, plainly: PepoChat publishes no accuracy or hallucination rate, and no vendor can honestly promise zero. Grounding means a wrong answer can often be traced to a wrong, missing or stale source you can fix, but the agent can still misread a passage: grounding reduces the risk, it does not remove it. You can start free with 100 AI replies a month and every chat feature on the free plan; paid plans start at $39 a month.
What to do next
- Give your bot an honest way to say it does not know with these chatbot fallback message examples.
- See how retrieval, instructions and handoff fit together in PepoChat on How answers work.
- Compare plans and reply allowances on the pricing page.
- Create a free PepoChat workspace, add your website, and ask it a question your site does not answer to see the handoff.
- Related explainers: What is RAG?, What is an LLM? and What is prompt injection?.
Frequently asked questions
- Is an AI hallucination the same as the AI lying?
- No. Lying requires knowing the truth and intending to deceive, and a language model has neither. It produces the most plausible continuation of the text, and sometimes the plausible answer is false. That is why NIST and many researchers prefer neutral words such as confabulation or error. The effect on the reader can be the same as a lie, so the output still needs checking.
- Does ChatGPT still hallucinate?
- Yes. In a September 2025 paper, OpenAI's own researchers wrote that hallucinations "are still present in the latest models", and every one of the 111 models on Vectara's public leaderboard, checked on 12 November 2026, produced some summaries that did not match the source article. Web search and attached documents reduce the problem without ending it, so open the sources for anything you will act on.
- Do AI image and video generators hallucinate?
- They can produce details that are not real, but the word matters less there. NIST's Generative AI Profile notes that confabulation is most commonly a problem for text, while for audio, image or video content, creative non-factual output can be the point. It becomes a problem when a generated image or clip is presented as a record of something that really happened.
- How do I stop ChatGPT from making things up in my own work?
- Give it the source text and tell it to answer only from that text, and to say "I don't know" when the text does not cover the question. Ask for the exact sentence it relied on, then find that sentence yourself. Ask factual questions twice in fresh chats, and check names, numbers, dates and citations against the original before you use them.
- Does an AI support agent hallucinate less than a general chatbot?
- It can, if it is grounded: an AI support agent that answers from your own knowledge base has your facts in front of it instead of guessing from training data. It can still misread a passage or repeat a stale page. PepoChat's agent answers from your knowledge base and offers a person when it finds nothing, but PepoChat publishes no hallucination rate, and no vendor can promise zero.
- What should a chatbot say instead of guessing?
- It should say plainly that it does not have that information and offer a next step, usually a person. PepoChat's agent, for example, is instructed to say "I don't have specific information about that in our knowledge base. Would you like me to connect you with a human support agent?" The handoff only helps if someone on your team actually reads the inbox.
Try this on your own site in ten minutes
PepoChat includes every chat feature on the free plan — 100 AI replies and 5 knowledge sources a month, no credit card.
Keep reading
Chatbase vs PepoChat: Which AI Support Agent Fits Your Team in 2026?
An honest side-by-side of Chatbase and PepoChat on pricing, free plan or trial, sources, handoff, booking, actions, channels and voice, and a pick by team type.
Intercom Fin Pricing Explained, and What a Flat-Plan Alternative Costs
What Intercom Fin's $0.99 per outcome adds up to once you count resolutions and seats, with a worked 2,000-conversation example and when a flat plan wins.