What Is AI Hallucination? (Why LLMs Make Things Up)

The word is a little unfair to the machine. A large language model is not lying — lying requires knowing the truth and choosing to say otherwise, and the model has no notion of truth to violate. As the standard reference frames it, a hallucination is a response that is presented as fact but is not justified by the model's training data or the input it was given. It is confidently, articulately wrong. That confidence is the dangerous part: the fabricated answer arrives in exactly the same authoritative tone as a correct one, with no flicker of doubt to warn you.

To see why hallucination is not a bug but a property, look at what an LLM actually does. It is a next-token predictor: given the text so far, it estimates a probability distribution over what word (token) comes next, samples one, appends it, and repeats. It was trained to make that continuation plausible — to sound like the human text it learned from — not to make it true. Fluency and truth are separate properties, and the training objective optimizes for fluency. Most of the time plausible and true coincide, which is why models are useful at all. Hallucination is what happens in the gap between them.

Three structural facts widen that gap. First, there is no built-in fact-checker — nothing in the forward pass cross-references a claim against reality before it is emitted. Second, there is a knowledge cutoff: the model knows nothing that happened after its training data ends, so a question about a library released last month lands in a void. Third, and most insidiously, the model has a strong bias toward completing the pattern rather than admitting a gap. As IBM's overview puts it, when the model lacks the information it needs it does not stop — it fills the space with the most plausible-sounding fabrication it can generate. Saying 'I don't know' is a rare shape in the training data; a confident answer is everywhere.

It helps to split hallucinations into two families, because they call for different defenses. A factual hallucination is when the model states something about the world that is simply false: a wrong date, a misattributed quote, a fabricated statistic, or — the developer's daily nightmare — a citation, API endpoint, or library function that does not exist. A faithfulness hallucination is narrower and sneakier: the model contradicts or drifts from the context you actually gave it. You paste a document and ask for a summary, and the summary asserts something the document never said. The first kind is a failure of world knowledge; the second is a failure to stay grounded in the input.

The single highest-leverage move is to stop relying on the model's memory and instead give it the real context. That is the whole idea behind retrieval-augmented generation (RAG): before the model answers, a retrieval step fetches the relevant trusted documents — from a knowledge base, a codebase, a vector database — and places them directly in the prompt. The model then answers from that material rather than from a fuzzy recollection. The original RAG paper by Lewis et al. showed this produces more specific and more factual output precisely because the generation is conditioned on retrieved evidence instead of parametric memory alone.

Grounding through tools is the other half. Instead of asking the model to recall a value, let it call a real function, hit a live API, query a database, or read an actual file — the space covered by MCP, APIs, and function calling. A model that can run a query does not have to guess the answer; a model that can read the file does not have to imagine its contents. This matters for agents especially: an AI agent that acts on hallucinated state compounds the error across every step, so grounding each step in a real observation is what keeps the loop honest.

Around those two pillars sit the everyday habits. Ask for citations you can actually check, and then check them — a fabricated source collapses the moment you follow the link. Lower the temperature for factual and code tasks so the model samples the most likely (usually most correct) continuation instead of a creative one. Verify outputs mechanically: compile the code, run the tests, resolve the import. And keep a human in the loop for anything consequential. When you write prompts, the same principle applies — hand the model the actual error, log, and code rather than expecting it to recall your system from a vague description.

Grounding works because it changes what the model is being asked to do. Ungrounded, the question is 'from everything you vaguely absorbed in training, what is the answer?' — an open-ended recall task where fabrication is a valid-looking completion. Grounded, the question becomes 'given these specific documents in front of you, what do they say?' — a reading task, where the retrieved or observed context constrains the generation. The model still predicts the next token, but now the highest-probability continuation is anchored to real material rather than free-floating in its parameters. You have narrowed the gap between plausible and true by supplying the truth.

Everything above lands squarely on the AI-fixes-bugs workflow. Ask a coding agent to fix a bug from a one-line description — 'the pricing page is broken' — and you have handed it a recall task. It has to guess which component, guess the state that triggered the failure, guess the API shape, and every guess is a chance to hallucinate a fix that patches a problem the code never had. That is the vague-description trap, and it is exactly the condition under which models fabricate.

BugMojo removes the guessing by grounding the agent in a real reproduction. The browser extension captures the failure with its surrounding state — an rrweb session replay, the console output, and the network request that fed the bad data — and hands that whole bundle to the coding agent (Claude Code, Cursor) over an MCP server. Now the question is no longer 'imagine what might be wrong'; it is 'here is the exact failing session, the exact error, and the exact response shape — fix this.' A grounded agent reading the actual reproduction is far less likely to hallucinate a fix than one inventing a cause from a sentence. Grounding is the antidote here too.