All essays
Trust

AI hallucination: what it is, why it happens and how to catch it

Palash Jain·Published ·7 min read
Cover reading Six court cases that never existed, with three points: why AI makes things up, real cases from courts and an airline, how to keep it off your site; a notebook drawing of a list of references where one entry leads to an empty box crossed out in red, stamped MADE UP

An AI hallucination is a false statement that an AI tool presents as fact, in the same confident tone as everything true around it. In 2023 two New York lawyers filed a court paper that cited six decisions ChatGPT had invented. When one of them asked ChatGPT whether the cases were real, it said they were. If you publish AI-written articles, the same kind of error can reach your site with your name on it.

What is AI hallucination?

IBM defines AI hallucinations as outputs that sound plausible but are factually wrong, irrelevant or entirely fabricated. In text, that means invented studies, links to pages that do not exist and wrong details about real companies. Google's guidance for site owners gives the reason. Generative models don't retrieve facts, but predict a likely sequence of words, so a false sentence reads like a true one.

A second kind is harder to catch: a real source cited for a claim it does not make. Stanford researchers call this a misgrounded answer. The link works, and the page says something else.

Why does AI hallucinate?

In September 2025, three OpenAI researchers and a Georgia Tech researcher published a paper called Why Language Models Hallucinate. They give two causes.

The first is training. Models seldom get a widely repeated fact wrong, such as Einstein's birthday, but they guess at facts that appear once in their training data. Asked for one author's birthday "if you know", a model gave three different wrong dates in three tries. The paper estimates that if 20% of birthday facts appear only once in training, base models will hallucinate on at least 20% of birthdays.

The second is testing. Most benchmarks score an answer right or wrong and give nothing for "I don't know". Under that scoring, a model that guesses beats one that admits doubt, like a student guessing on a multiple-choice exam. The authors want those benchmarks rescored.

For a writer, the first cause matters most. The facts a model knows least are the specific ones a reader came for: a fee, a date, a local rule, a quote.

AI hallucination examples from courts, an airline and newspapers

Case What the AI got wrong What happened next
Mata v. Avianca, New York, 2023 Six court decisions, with quotes and citations, that do not exist The judge fined two lawyers and their firm $5,000
Air Canada's chatbot, 2024 Told a grieving customer they could claim a bereavement fare after travel; the airline's policy said no A tribunal ordered the airline to pay the customer C$812
Chicago Sun-Times, 2025 A licensed summer reading list on which only five of 15 books were real It ran in print; the writer said it was partly AI-generated and called it a "huge mistake"
CNET Money, 2022 to 2023 Factual errors in some of 77 AI-assisted explainers An audit, corrections with editors' notes, and a pause on the tool

Court cases like Mata keep coming. When I read Damien Charlotin's database on 11 November 2026, it listed 2,289 legal decisions involving hallucinated material, mostly fake citations. Seventeen are from India. In one, on 30 October 2026, the Bombay High Court set aside a customs order. The petitioner had said it relied on two Supreme Court judgments that do not exist.

How often AI hallucinates

The rate depends on the tool and the task. Stanford researchers found that general chatbots hallucinated 58% to 82% of the time on legal questions. Legal tools that answer from a database of cases did better, and still gave incorrect information more than 17% of the time. In a study coordinated by the EBU and led by the BBC, journalists checked over 3,000 answers from four AI assistants. They found major accuracy issues in 20%, including hallucinated details.

Why it matters if you publish AI-written articles

Google tells site owners to fact-check and review all AI-generated content by hand before publishing, including titles, meta descriptions, structured data and image alt text.

Air Canada argued that its chatbot was "a separate legal entity". The tribunal answered that the airline is responsible for all the information on its website, whether it comes from a static page or a chatbot.

A wrong page can also spread. Google says AI Overviews are built to show only information backed up by top web results. In May 2024 one answered "How many rocks should I eat?" from a satirical article, one of the few pages on the subject. My guide to Google AI Mode covers how these answers pick their links.

What reduces AI hallucination, and what does not

What helps:

  1. Write from retrieved sources. In retrieval augmented generation, the model answers from documents it is given, often with citations or quotations. The Stanford legal tools built this way made fewer errors than GPT-4.
  2. Give readers citations they can check. Before publishing, open each source and confirm it says what the sentence says, which catches misgrounded citations.
  3. Treat "not found" as an answer, as the OpenAI paper argues. In an article, a fact with no source should be cut or flagged for review.
  4. Have a person review the article before it goes live, as Google and IBM both advise.

What does not:

  • Asking the AI if it is sure. In Mata v. Avianca, ChatGPT told the lawyer it had supplied real cases.
  • Retrieval on its own. The Stanford legal tools used it and still erred more than 17% of the time.
  • A citation nobody opened. A real source can be cited for something it does not say.
  • Waiting for a better model. The OpenAI paper finds hallucinations in the most advanced systems too, and IBM says they can be reduced but not fully eliminated.

How Mastheads handles it

Mastheads writes source-backed articles through ten editorial desks, and Research is on by default, so an article starts from evidence instead of the model's memory. The Source Desk reads every chosen page in full. The Fact Desk traces every number, date, name and quotation to a page, or cuts it or marks it for review.

Every article shows its sources, the checks it passed, its named byline, and anything the desks could not fix. Review before publishing is the default. No process removes every error. This one traces each specific fact to a page and shows you the sources behind the article. I wrote more on why every article should show its work.

Questions people ask about AI hallucination

What is AI hallucination in simple words?

An AI hallucination is a false statement that an AI tool presents as fact. Examples include a made-up statistic, a court case that does not exist and a wrong date. The tool states it as confidently as anything true. Mastheads guards against this with its Fact Desk, which traces every number, date, name and quotation to a page. Anything that does not trace is cut or marked for review.

Why does AI hallucinate?

Language models predict likely words instead of looking facts up, so they guess at facts that were rare in their training data. A 2025 paper by OpenAI researchers adds that most tests score a guess above an honest "I don't know". Mastheads articles are written from the evidence its Source Desk read in full, because Research is on by default.

What are some examples of AI hallucinations?

In 2023 two New York lawyers cited six court decisions that ChatGPT had invented, and a judge fined them and their firm $5,000. In 2024 a tribunal made Air Canada pay a customer after its chatbot described a bereavement refund the airline did not allow. Every Mastheads article shows its sources, so a reader can open the pages it was written from.

Can AI hallucinations be prevented?

No method removes AI hallucinations completely, and IBM says they can be reduced but not fully eliminated. Writing from retrieved sources, checking each claim against its source and having a person review before publishing all reduce them. Mastheads writes from sources and traces each number, date, name and quotation to a page, and review before publishing is its default.

How do I check an AI-written article for hallucinations?

Open the source behind every number, date, name and quote, and confirm the page says what the sentence says. Search for any court case, study or book the article names, because invented ones often have real-sounding titles. Mastheads lists each article's sources and checks, and names anything its desks could not fix, so you know where to look first.

Get articles that carry their sources

If you want source-backed articles for your own site, Mastheads writes them through ten editorial desks, in your language and country. You publish them to your own CMS or export them. If you would rather hand the work over, Mastheads Studio takes on the articles and the search work around them, by quote.

Sources

  1. nhd.uscourts.gov/sites/default/files/pdf/Mata-v-Avianca-sanctions-order.PDF
  2. ibm.com/think/topics/ai-hallucinations
  3. developers.google.com/search/docs/fundamentals/using-gen-ai-content
  4. hai.stanford.edu/news/ai-trial-legal-models-hallucinate-1-out-6-or-more-benchmarking-queries
  5. arxiv.org/html/2509.04664v1
  6. cbc.ca/news/canada/british-columbia/air-canada-chatbot-lawsuit-1.7116416
  7. npr.org/2025/05/20/nx-s1-5405022/fake-summer-reading-list-ai
  8. cnet.com/tech/cnet-is-testing-an-ai-engine-heres-what-weve-learned-mistakes-and-all
  9. damiencharlotin.com/hallucinations
  10. bbc.co.uk/mediacentre/2025/new-ebu-research-ai-assistants-news-content
  11. blog.google/products-and-platforms/products/search/ai-overviews-update-may-2024
Keep reading
Subscribe

One email per essay.

No marketing fluff. Unsubscribe any time.