Jargon Buster / Hallucination
Hallucination.
AI & AutomationWhen an AI model states something false or invented with complete confidence.
What is an AI hallucination?
A hallucination is when an AI model produces information that sounds confident and plausible but is false or made up, such as an invented statistic, a non-existent source or a policy your business does not have.
It happens because language models generate the most likely-sounding text. They have no built-in sense of whether a statement is true, so a fluent wrong answer and a fluent right answer look identical.
Why it matters
A hallucination in an internal draft is an annoyance. In a customer email, a quote, a contract summary or a chatbot reply, it is a legal and reputational problem. An invented price, a misquoted regulation or a made-up reference can cost you a client or land you in a dispute, and 'the AI said it' is no defence. Any business using AI in front of customers needs a plan for catching errors before they leave.
Why it happens
- Prediction over knowledge The model predicts plausible words; it does not look facts up unless it is connected to a source.
- Missing source material Ask about your own prices or policies without supplying them and it will fill the gap.
- Pressure to answer Models tend to give an answer rather than say they do not know, unless told otherwise.
- Vague instructions Loose prompts leave room for the model to improvise detail.
The honest view
Hallucinations can be reduced a great deal, but no current model eliminates them. Newer models are more accurate and also more fluent, which makes the remaining errors harder to spot. The real failure is usually a process with nobody checking, rather than the model itself.
What works: give the model the source material, tell it to answer only from that material, allow it to say 'I don't know', and ask it to quote the passage it relied on. Requiring a citation is the most underrated habit, because a checker can confirm a quote in seconds.
What to do about it
Sort your AI uses by risk. Anything involving prices, numbers, names, legal or medical claims, or promises to customers should be checked by a person against the original source before it goes out. Add a line to your standard prompts telling the model to say when it is unsure. For customer-facing tools, ground answers in your own approved documents and test them with questions designed to tempt the model into guessing.
Related terms.
Retrieval-Augmented Generation (RAG)
A method where AI first looks up your trusted documents, then writes its answer from what it finds.
AI & AutomationSee more HHuman in the Loop
A design where a person reviews or approves an AI system's work at set points before it takes effect.
AI & AutomationSee more LLarge Language Model (LLM)
An AI system trained on huge amounts of text that can write, summarise, answer questions and follow instructions.
AI & AutomationSee more EE-E-A-T
Experience, Expertise, Authoritativeness and Trustworthiness: the qualities Google looks for in credible content.
Search & AI VisibilitySee moreGot a question about your own setup? Let's talk.
Your first call is with our founder, Rhodri Williams.