When ChatGPT insists that the Eiffel Tower was “moved to Marseille in 2024,” that’s not AI creativity—it’s a hallucination. Hallucinations are false statements generated confidently by an AI. In academic terms, they’re a side effect of the probabilistic nature of language models. Unlike humans who pause when unsure, models are designed to always answer—even when guessing. probabilistic nature always answer There are two major types: Factual hallucination: Outputs contradict reality (e.g., “honey helps diabetics lower blood sugar”).
Faithfulness hallucination: Outputs drift from the user’s intent (e.g., answering about nutrition when asked about diabetic sugar substitutes). Factual hallucination: Outputs contradict reality (e.g., “honey helps diabetics lower blood sugar”). Factual hallucination: Faithfulness hallucination: Outputs drift from the user’s intent (e.g., answering about nutrition when asked about diabetic sugar substitutes). Faithfulness hallucination: Why Hallucinations Happen: Four Core Mechanisms 1. Bad Data, Bad Knowledge Bad Data, Bad Knowledge Data is the foundation of every model’s “understanding.”When training data is incomplete, outdated, or biased, the model simply guesses. In long-tail or low-resource topics—say, a rare medical syndrome—the model might never have seen relevant data, so it fills the gap using probability patterns from unrelated text. That’s where “zero-resource hallucinations” emerge. Duplicate data also reinforces noise. If a flawed statement (“Vitamin C cures COVID”) appears often enough, the model begins to treat it as statistically true. 2. Training Objectives That Reward Guessing Training Objectives That Reward Guessing Early LLM training rewarded “getting something right” but didn’t penalize “being confidently wrong.”That’s like an exam where students earn points for right answers but lose nothing for wild guesses. This binary scoring makes models reckless. They’d rather fabricate an answer than stay silent. OpenAI discovered this during early fine-tuning: models often “invented” nonexistent functions or events simply because their training never encouraged saying “I don’t know.” “I don’t know.” Modern approaches—like Anthropic’s constitutional AI and DeepMind’s triple reward systems—now introduce “I don’t know” as a valid, even rewarded, behavior. constitutional AI triple reward systems “I don’t know” 3. Architectural Limits: The Attention Dilution Problem Architectural Limits: The Attention Dilution Problem Transformer-based models still struggle with long-term reasoning.When processing huge contexts, their attention mechanism dilutes key information. The result: logical drift. Ask an LLM how Antarctic ice melt affects African agriculture—it might generate poetic nonsense because it fails to hold distant causal links in memory. New architectures like Gemini 2.5 Pro’s hybrid attention (Transformer + PathFormer) aim to counteract this, dynamically balancing global and local focus. But architectural hallucination is far from solved. Gemini 2.5 Pro’s hybrid attention 4. Inference Chaos: Small Errors, Big Consequences Inference Chaos: Small Errors, Big Consequences During generation, each token depends on the one before it.If the model mispredicts even once (“Paris → Berlin”), that error cascades—creating what researchers call a “logic avalanche.” “logic avalanche.” Since models lack built-in verification, they can’t double-check their own claims.As one NUS paper put it: “A language model cannot know when it doesn’t know.” Fixing the Glitch: How We’re Learning to Control Hallucinations 1. Clean, Smart Data Better data beats bigger data. Projects like Concept7 evaluate concept familiarity during training—helping models know when they’re venturing into the unknown.Meanwhile, DeepSeek V3 reduces hallucination rates by 17% through real-time web grounding—keeping data fresh via live search integration. Concept7 concept familiarity DeepSeek V3 real-time web grounding Google’s Gemini filters redundant and contradictory content, while OpenAI experiments with generating synthetic counterexamples—feeding the model “tricky” fake data to improve fact-check resilience. Gemini OpenAI 2. Smarter Architecture: Teaching Models to Think, Not Guess Gemini 2.5’s multi-stage reasoning pipeline includes: Hypothesis generation
Dynamic thinking depth (more steps for harder questions)
Hybrid attention
Closed-loop verification Hypothesis generation Hypothesis generation Dynamic thinking depth (more steps for harder questions) Dynamic thinking depth Hybrid attention Hybrid attention Closed-loop verification Closed-loop verification It’s like giving the model a built-in Socratic method—ask, reason, verify, repeat. This “thinking in public” design also enables observable reasoning chains through APIs, letting developers pinpoint where logic went off-track. observable reasoning chains 3. Reward Redesign: Make “I Don’t Know” a Win The biggest behavioral shift comes from how we score models. A new three-tier system changes the game: three-tier system Behavior

Reward

Example



Correct answer

+1

“The capital of France is Paris.”



Admit uncertainty

+0.5

“I’m not sure, but I can check.”



Confidently wrong

-1

“The capital of France is Berlin.” Behavior

Reward

Example



Correct answer

+1

“The capital of France is Paris.”



Admit uncertainty

+0.5

“I’m not sure, but I can check.”



Confidently wrong

-1

“The capital of France is Berlin.” Behavior

Reward

Example Behavior Behavior Reward Reward Example Example Correct answer

+1

“The capital of France is Paris.” Correct answer Correct answer +1 +1 “The capital of France is Paris.” “The capital of France is Paris.” Admit uncertainty

+0.5

“I’m not sure, but I can check.” Admit uncertainty Admit uncertainty +0.5 +0.5 “I’m not sure, but I can check.” “I’m not sure, but I can check.” Confidently wrong

-1

“The capital of France is Berlin.” Confidently wrong Confidently wrong -1 -1 “The capital of France is Berlin.” “The capital of France is Berlin.” This “anti-hallucination” scoring led to hallucination drops of 30–70% in medical and legal tasks.Even Anthropic’s Claude 3.5 now includes a “decline to answer” function—turning AI modesty into accuracy. 30–70% Claude 3.5 4. Uncertainty Detection: The SELF-FAMILIARITY Approach Before generating, a model can now check its own familiarity with the topic. check its own familiarity Here’s a mini-implementation using Python + fuzzy matching to detect unknown concepts: from fuzzywuzzy import fuzz

def detect_unknown_concepts(text, known_concepts, threshold=0.6):
    unknowns = []
    for word in text.split():
        if len(word) < 3: continue
        if max(fuzz.ratio(word.lower(), k.lower()) for k in known_concepts) < threshold * 100:
            unknowns.append(word)
    return set(unknowns)

medical_terms = ["glucose", "insulin", "carbohydrate", "diabetes"]
output = "Honey contains glycetin, which helps regulate diabetes."
print(detect_unknown_concepts(output, medical_terms)) from fuzzywuzzy import fuzz

def detect_unknown_concepts(text, known_concepts, threshold=0.6):
    unknowns = []
    for word in text.split():
        if len(word) < 3: continue
        if max(fuzz.ratio(word.lower(), k.lower()) for k in known_concepts) < threshold * 100:
            unknowns.append(word)
    return set(unknowns)

medical_terms = ["glucose", "insulin", "carbohydrate", "diabetes"]
output = "Honey contains glycetin, which helps regulate diabetes."
print(detect_unknown_concepts(output, medical_terms)) Result: {'glycetin'} {'glycetin'} In real systems, such detection triggers a refusal response or re-verification step. refusal response 5. Retrieval-Augmented Generation (RAG): Marrying Memory and Search When models don’t know something—let them look it up. look it up. RAG combines retrieval with generation. Below is a simplified LangChain + Chroma example that turns PDFs into searchable AI knowledge bases: from langchain.document_loaders import PyPDFLoader
from langchain.vectorstores import Chroma
from langchain.chains import RetrievalQA
from langchain.embeddings import OpenAIEmbeddings
from langchain.llms import OpenAI

loader = PyPDFLoader("medical_paper.pdf")
docs = loader.load()

vectorstore = Chroma.from_documents(docs, OpenAIEmbeddings(), persist_directory="./db")
qa = RetrievalQA.from_chain_type(
    llm=OpenAI(temperature=0),
    chain_type="stuff",
    retriever=vectorstore.as_retriever(search_kwargs={"k": 3})
)

query = "Can honey replace sugar for diabetics?"
answer = qa({"query": query})
print(answer["result"]) from langchain.document_loaders import PyPDFLoader
from langchain.vectorstores import Chroma
from langchain.chains import RetrievalQA
from langchain.embeddings import OpenAIEmbeddings
from langchain.llms import OpenAI

loader = PyPDFLoader("medical_paper.pdf")
docs = loader.load()

vectorstore = Chroma.from_documents(docs, OpenAIEmbeddings(), persist_directory="./db")
qa = RetrievalQA.from_chain_type(
    llm=OpenAI(temperature=0),
    chain_type="stuff",
    retriever=vectorstore.as_retriever(search_kwargs={"k": 3})
)

query = "Can honey replace sugar for diabetics?"
answer = qa({"query": query})
print(answer["result"]) By grounding output in retrieved documents, Gemini’s “search anchoring” feature has cut factual errors down to 0.7%. 0.7% Real-World Hallucination Fixes: How Tech Giants Are Fighting Back OpenAI: Rewarding Honesty OpenAI: OpenAI’s new model behavior framework shifts scoring from “accuracy only” to “accuracy + honesty.”Models get positive reinforcement for acknowledging knowledge gaps, monitored by an internal Hallucination Leaderboard. Hallucination Leaderboard Google Gemini: Search Grounding Google Gemini: Gemini’s API uses google_search_retrieval() to pull fresh data, attaching metadata such as source, confidence score, and timestamp—keeping outputs current and verifiable. google_search_retrieval() Anthropic Claude: Legal Citations That Hold Up in Court Anthropic Claude: Claude’s “Quote & Verify” pipeline first extracts source quotes, then verifies them before responding.If the citation can’t be validated, it simply says: *“Unable to confirm this information.”*This system cut legal hallucinations to 0%, earning federal court endorsements in the U.S. 0% Evaluating Hallucinations: Benchmarks and Metrics Model

General Hallucination Rate

Factual Tasks



DeepSeek V3

2%

29.7%



DeepSeek R1

3%

22.3%



Gemini 2.0

0.7%

0.7%



GPT-4o

1.5%

1.5% Model

General Hallucination Rate

Factual Tasks



DeepSeek V3

2%

29.7%



DeepSeek R1

3%

22.3%



Gemini 2.0

0.7%

0.7%



GPT-4o

1.5%

1.5% Model

General Hallucination Rate

Factual Tasks Model Model General Hallucination Rate General Hallucination Rate Factual Tasks Factual Tasks DeepSeek V3

2%

29.7% DeepSeek V3 DeepSeek V3 2% 2% 29.7% 29.7% DeepSeek R1

3%

22.3% DeepSeek R1 DeepSeek R1 3% 3% 22.3% 22.3% Gemini 2.0

0.7%

0.7% Gemini 2.0 Gemini 2.0 0.7% 0.7% 0.7% 0.7% 0.7% 0.7% GPT-4o

1.5%

1.5% GPT-4o GPT-4o 1.5% 1.5% 1.5% 1.5% 1.5% 1.5% Tools like MiniCheck (400× cheaper than GPT-4 for fact verification) and FENCE (declaration-level fact scoring) are now standard in research pipelines.The new 2025 international evaluation framework even includes “wrong answers get penalties, refusals earn points.” MiniCheck FENCE “wrong answers get penalties, refusals earn points.” The Hard Truth: You Can’t Fully Eliminate Hallucinations Even OpenAI admits that hallucinations are mathematically inevitable.But that doesn’t mean they’re unmanageable. mathematically inevitable Future LLMs will balance accuracy and creativity via probabilistic governance—“hallucination masks” that dynamically control uncertainty.Meanwhile, lightweight local evaluators like MiniCheck-FT5 make hallucination detection practical even for small orgs. accuracy and creativity MiniCheck-FT5 As one Google researcher put it: “The goal isn’t zero hallucination. It’s knowing when to doubt the machine.” “The goal isn’t zero hallucination. It’s knowing when to doubt the machine.” Final Thoughts We’re entering a new era where AI systems must justify what they say, not just say it confidently.The next breakthroughs won’t come from bigger models—but from smarter governance: data hygiene, architecture reform, reward redesign, and self-awareness. justify If hallucinations are AI’s dreams, the job of engineers is not to wake it up—but to teach it to know when it’s dreaming.

Glitch

DeepMind

Google

OpenAI

Spring AI RAG, Demystified: From Toy Demos to Production-Grade Retrieval

From Black Box to Transparent: How to Read the Inner Logic of LLMs

Read My Stories

Developer

Too Long; Didn't Read

The Science of AI Hallucinations—and How Engineers Are Learning to Curb Them

The Science of AI Hallucinations—and How Engineers Are Learning to Curb Them

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

A Developer's Guide to Merging AI with the Spring Ecosystem

The Noonification: How Often Do NFTs Pass The Howey Test? (1/13/2023)

Darwin's Hybrid Intelligence to Align AI & Human Goals for Startups & VCs

The Noonification: White Man (11/26/2022)

The Noonification: The Metaverse is a Sh*tshow (11/2/2022)

100 Days of AI Day 1: From Newsletter to Podcast, Leveraging AI for Audio Transformation

A Developer's Guide to Merging AI with the Spring Ecosystem

The Noonification: How Often Do NFTs Pass The Howey Test? (1/13/2023)

Darwin's Hybrid Intelligence to Align AI & Human Goals for Startups & VCs

The Noonification: White Man (11/26/2022)

The Noonification: The Metaverse is a Sh*tshow (11/2/2022)

100 Days of AI Day 1: From Newsletter to Podcast, Leveraging AI for Audio Transformation

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps