What is anAI hallucination?
Language models keep writing fluently even when the knowledge runs out, and that is where invented claims come from.
Get in touch
Let's talk about your project.
First we check whether the project fits your business model. Then you get a proposal with phases and effort.
Talk about AI in your team or call: +49 151 1576 5566An AI hallucination is a claim an AI invents and phrases as if it were backed up. Because a language model only knows what was in its training data, it can fill a gap with an answer that sounds plausible and is still wrong.
- In a Bitkom survey from November 2025, 42 percent of people who use AI for search said they had already received false or invented information.
- The tricky part is that an invented claim sounds just as sure of itself as a correct one.
- Looking things up in your own documents with RAG helps, but it does not solve the problem completely.
Why does an AI hallucinate, and what does it mean for your content?
A large language model writes from what it saw in training. Ask it for last quarter's parts list, or for a figure that never appeared anywhere, and it may admit it does not know. More often than anyone would like, though, it puts something together that reads just as confidently as a correct answer. The model matters too: in the model card for its open gpt-oss models, OpenAI itself states that smaller models have less world knowledge and are more prone to making up facts (remarkably frank for a vendor, we think).
How far RAG helps you keep hallucinations in check
With RAG, the AI checks your own documents before answering and names its source. That makes invented claims rarer but does not prevent them. In a preregistered 2024 Stanford study, legal research tools whose vendors had promoted RAG got between a sixth and a third of the test queries wrong or misleading, depending on the tool. An answer with a citation is checkable, then, but not automatically correct.
How to protect your business from the fallout of an AI hallucination
Anything that goes out, whether a figure, a commitment or a price, gets looked at by someone on your team first. Your people should also know where the AI tends to go wrong, and Article 4 of the EU AI Act asks for exactly that: since 2 February 2025 it has required businesses that use AI to take measures for their staff's AI literacy. Hallucinations can affect you even without AI of your own. Whether an AI system describes your company correctly, in shortened form, wrongly or with invented details can be checked by comparing each statement with your documents.
Frequently asked questions
Can I rely on tools that promise to be hallucination-free?
We would not. In the Stanford study some vendors had made exactly that promise, and their tools still got part of the test queries wrong.
Does my team need training on hallucinations?
Yes. The European Commission names invented content as a risk staff should be told about, for example when they use ChatGPT for marketing copy or translations. Article 4 does not require a certificate for it.
Terms you should know in the same context
RAG (retrieval augmented generation) · LLM (large language model) · Local AI · AI literacy (Article 4) · AI assistant · Back to the AI glossary A to Z
An AI that can be wrong is no reason to do without it, but it is a reason to look at its answers before they go out.