In June 2023, a federal judge in New York fined two lawyers and their firm $5,000. Their brief cited six court decisions that didn't exist. One of the lawyers had found them by asking an AI chatbot. That's an AI hallucination, an answer that sounds sure of itself and is simply false.
You don't need to be a lawyer for this to matter. Students can get invented sources in a reading list. Teachers can get a lesson summary built on a study that was never run. Writers can get quotes nobody said.
Below you'll find what an AI hallucination is, real cases from courts and research, and the reasons large language models make things up. Then we walk through a routine for checking an AI answer in a few minutes.
What is an AI hallucination?
An AI hallucination is content an AI system produces that is false, unsupported or invented, yet presented as if it were true. Common forms are made-up facts, fake quotes, wrong dates and citations to articles or court cases that don't exist. The US National Institute of Standards and Technology (NIST) calls this "confabulation" and describes it as AI systems that "confidently present erroneous or false content." The confidence is the problem. The wrong answer reads exactly like a right one.
The word "hallucination" is a metaphor, and some researchers dislike it. NIST's profile records a concern from commenters that words like "hallucination" make AI sound more human than it is. You'll also see "fabrication" and "confabulation" used for the same thing.
What do AI hallucination examples look like in real life?
The clearest AI hallucination examples come from courts and research, because there someone checks every source. Here are three well-documented cases and one worked example.
The lawyer and the fake airline cases
In Mata v. Avianca, a lawsuit brought by a passenger whose knee was hit by a serving cart on a flight, heard in the US District Court for the Southern District of New York, lawyers filed a brief citing cases such as "Varghese v. China Southern Airlines." The cases didn't exist. One lawyer later said he had asked the chatbot whether Varghese was a real case, and it said yes.
On 22 June 2023, Judge P. Kevin Castel fined the two lawyers and their firm $5,000. He also ordered them to send letters to their client and to each real judge falsely named as the author of the fake opinions. His opinion called relying on fake opinions "an abuse of the adversary system."
Fake references in research summaries
In 2023, William Walters and Esther Isabelle Wilder asked two versions of a popular chatbot to write 84 short literature reviews. They checked all 636 references. Of the references from the older model, 55% were fabricated. For the newer model, 18% were. Many real references also had errors. That was 43% for the older model and 24% for the newer one.
A smaller 2023 study in Cureus by Mehul Bhattacharyya and colleagues asked a chatbot to write 30 short medical papers. Of 115 references, 47% were fabricated, 46% were real but inaccurate, and only 7% were real and accurate.
A worked example you can practice on
Here's an invented example of the kind of answer a student might get when asking for evidence about note-taking. The study, journal and number are all made up for this post.
Every highlighted part is a claim you can check. Search the journal name and you'd find nothing. That's the pattern to learn. Fake citations usually look complete, with authors, a year, volume, issue and pages. Completeness proves nothing.
Why does AI hallucinate?
AI hallucinates because a language model is built to produce likely text, and likely isn't the same as true. When it doesn't know an answer, the most fluent continuation is often a plausible guess, so it guesses.
It predicts the next word
UNESCO's 2023 guidance on generative AI in education describes the basic process. The model is trained on text from webpages and other online media, estimates the probability of words in context, and uses those probabilities to predict the next likely word. There's no step where it checks a fact against a database. A citation is just a pattern, author names plus a year plus a journal-sounding title, and the model is very good at patterns.
Gaps and errors in training data
A 2024 survey in ACM Transactions on Information Systems by Lei Huang and colleagues groups the causes into three areas. Data problems include errors in the training text and gaps around rare or recent topics. Training problems include models learning to please the user. Inference problems include the way text is sampled and models being overconfident.
Tests reward guessing
In a 2025 paper, Adam Tauman Kalai, Ofir Nachum, Santosh Vempala and Edwin Zhang compared language models to students taking a test. If a wrong answer and "I don't know" both score zero, guessing is the better strategy. They argue that many benchmarks score models this way, which pushes models to guess when they're unsure.
What are the main types of AI hallucination?
Researchers usually split hallucinations into two families. Factuality hallucinations conflict with real-world facts. Faithfulness hallucinations ignore or contradict what you gave the model, such as your instructions or a document you pasted.
Type | What it looks like | Example | How to catch it |
|---|---|---|---|
Fabricated source | A reference, case or URL that doesn't exist | A journal article with real-sounding authors | Search the exact title and DOI |
Wrong fact | A real thing with a wrong detail | The right book with the wrong author | Check one reliable source |
Made-up quote | Words attributed to someone who never said them | A quote from a scientist with no page number | Find the words in the original |
Overclaim | A claim stated as always true when it isn't | "Studies prove" with no study named | Ask what the evidence is |
Context error | A summary that contradicts the text you supplied | A summary of your essay with a point you didn't make | Reread your own source |
Logic error | Steps that don't add up | A sum done right but the conclusion wrong | Redo the reasoning yourself |
The table loosely follows the Huang survey. The first four rows are problems with facts, and the last two are problems of faithfulness to what you supplied.
How do you fact-check an AI answer?
Treat every specific claim as unverified until you find it in a source you can open. Library guides from universities give the same core advice. Purdue University Northwest's library guide warns you not to assume a source exists just because the AI gave a complete-looking citation.
List the claims. Underline every name, number, date, quote and reference in the answer.
Find each source. Search the exact title in a library database or a scholar search. If you can't find it in two places, treat it as fake.
Read the original. A real source can still be misused. Check that it says what the answer claims, on the page given.
Read laterally. Leave the answer and see what other reliable sources say about the same claim. Government agencies and professional bodies are good for specialist facts.
Check every quote. Find the exact wording in the original. If you can't, don't use it as a quote.
Keep a record. Note what you checked and what you changed. If your course allows AI, this record helps you cite and disclose AI use honestly.
Which mistakes make hallucinations more likely to slip through?
Most hallucinations get through because nobody checks. These are the habits that let them in.
Trusting the format. A reference in perfect APA style can still be invented. Volume, issue and page numbers are easy to generate.
Checking only the title. Walters and Wilder found many real references with wrong details. A real title with the wrong year or authors is still a citation error.
Asking for sources after the answer. If you ask the tool to "add references" to text it already wrote, you invite it to fit sources to claims.
Using AI on niche or recent topics. The Huang survey lists gaps in training data around rare and recent knowledge as a main cause.
Copying numbers without context. An oddly exact figure like "34%" with no named study is a warning sign.
Can an AI detector find hallucinations?
No. An AI detector estimates whether text reads like AI writing. It doesn't check whether the facts are true. A fully human essay can have a wrong date, and an AI-written answer can be accurate. For facts, you need sources.
The two checks answer different questions. A detector score tells you about style, and you can read how AI detectors work to see what they measure. Fact-checking tells you whether a claim holds up. For a teacher, a reference that doesn't exist is concrete evidence that's easy to discuss with a student. Go through the sources together and ask how each one was found.
How can you reduce hallucinations when you use AI?
You can lower the risk but not remove it. Give the tool better material to work from, ask narrower questions, and plan to check the output anyway.
Give it the source. Paste the article or chapter and ask questions about that text. Then compare the answer with the text, since faithfulness errors still happen.
Ask it to say when it doesn't know. This can help, though Kalai and colleagues show models are pushed toward guessing.
Find sources yourself. Use the library to find references, and use AI, if allowed, for brainstorming or explaining ideas.
Ask narrow questions. "Summarize section 2 of this report" is easier to check than "Tell me everything about this topic."
These habits are part of AI literacy, and they get faster with practice. You can paste a draft into our free AI detector to see which sentences read most like AI writing. It won't check facts, so check every claim and source by hand too.
Frequently asked questions
What is an AI hallucination in simple terms?
It's when an AI tool makes something up and presents it as true. The answer might include a fake statistic, a quote nobody said, or a book or court case that doesn't exist. It reads just as confidently as a correct answer, which is why it's easy to miss. The safe habit is to check any specific claim against a real source before you use it.
Why does AI make up fake citations?
A citation is a predictable pattern of author names, a year, a title, a journal and page numbers. Language models are good at producing that pattern without retrieving a real record. When the model doesn't have a matching source, it can still produce something with the right shape. Studies in 2023 found that many AI-generated references were fabricated, and many real ones had wrong details.
Are newer AI models free of hallucinations?
No. Newer models can make fewer errors, and Walters and Wilder found a drop from 55% to 18% fabricated references between two versions of one chatbot. Eighteen percent is still nearly one in five. Researchers also argue that the way models are tested rewards guessing over admitting uncertainty, so some hallucination is likely to remain.
How can I tell if a source from AI is real?
Search the exact title in your library catalogue, a scholar search or the journal's own website. Check the DOI if one is given, since a fake DOI usually leads nowhere or to a different paper. Confirm the authors, year and journal match. If you can't find it in two places, don't cite it.
Is it plagiarism to use a hallucinated source?
It's closer to fabrication. The US Office of Research Integrity lists fabrication next to plagiarism as research misconduct, and many school policies treat invented sources as a serious breach. Citing a source you never read misleads your reader either way. Our guide on whether using AI is plagiarism covers how schools tend to treat AI use, and checking every source avoids the problem completely.
Do AI hallucinations happen in images too?
Yes. The University of Illinois library guide on generative AI uses an example from images. Image tools sometimes add extra fingers to hands, because the model has picked up the pattern of fingers without understanding how a hand is built. The same rule applies. Don't treat AI output as a record of reality.
Sources
Mata v. Avianca, Inc., Opinion and Order on Sanctions, 678 F.Supp.3d 443, US District Court, S.D.N.Y., 22 June 2023
Fabrication and errors in the bibliographic citations generated by ChatGPT, Walters and Wilder, Scientific Reports, September 2023
High Rates of Fabricated and Inaccurate References in ChatGPT-Generated Medical Content, Bhattacharyya et al., Cureus, May 2023
Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (NIST AI 600-1), National Institute of Standards and Technology, July 2024
A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions, Huang et al., ACM Transactions on Information Systems, 2024
Why Language Models Hallucinate, Kalai, Nachum, Vempala and Zhang, arXiv, September 2025
Guidance for generative AI in education and research, UNESCO (Miao and Holmes), 2023
Definition of Research Misconduct, US Office of Research Integrity, HHS, accessed October 2026
Fact-Checking AI-Generated Information, Purdue University Northwest Library, updated August 2026
Hallucinations (Introduction to Generative AI), University of Illinois Urbana-Champaign Library, updated April 2026
Comments