The Hallucination Rate Your Team Quotes Is a Guess

A hallucination rate is a measurement only when it comes with a test set, a written scoring rule and a margin of error. Without all three it is a guess, and a guess should not decide whether a retrieval-augmented generation (RAG) system goes live.

What follows is a composite of a common pattern. The numbers are illustrative, not one client’s data. A product manager tells the steering committee that the assistant hallucinates “about 20 percent of the time”. The figure comes from a few dozen questions typed in by hand. Later the team builds a proper test set of 2,000 real user questions, scores every claim in every answer against its source passage, and finds at least one unsupported claim in 38 percent of answers.

My position is blunt. A hallucination rate with no test set behind it should not be presented to a steering committee at all. “We haven’t measured it” is a more useful answer than a confident 20 percent, because it tells the room what to fund next.

What happened when researchers measured the “hallucination-free” tools?

The best public example comes from legal research. LexisNexis had marketed its Lexis+ AI tool as delivering “100% hallucination-free linked legal citations”. In 2024 Varun Magesh and colleagues at Stanford wrote 202 legal questions, a third of them tied to a jurisdiction or a date and one in nine resting on a false premise, registered them in advance so they could not be chosen to flatter or embarrass anyone, and put them to Lexis+ AI, Thomson Reuters’ Westlaw AI-Assisted Research and Ask Practical Law AI, and GPT-4. The team checked every answer by hand and reported 95 percent confidence intervals.

In the versions tested in spring 2024, Lexis+ AI and Ask Practical Law AI each hallucinated on 17 percent of the questions. Westlaw’s tool hallucinated on 33 percent, and GPT-4 without retrieval on 43 percent. Retrieval helped. It did not get anywhere near zero. With that many hard questions, these are stress-test rates rather than a forecast of everyday use.

The more useful finding is about definitions. The vendor’s claim held only in a narrow sense: the citations linked to real documents. The researchers counted an answer as a hallucination if it was false, or if it cited a source that did not support what it said. A tool can link to a real case and still misstate it. Both sides were quoting a hallucination rate. Only one published its test set and its rule, and the study was published in the Journal of Empirical Legal Studies in 2025.

Why can’t thirty questions tell you the rate?

A guessed rate usually fails in three ways at once. It uses too few questions, the wrong questions and no agreed definition.

Start with the number of questions. If six of 30 hand-typed answers look wrong, the measured rate is 20 percent, but the Wilson score interval, a standard way to put a 95 percent margin of error on a proportion, runs from about 10 to 37 percent. The same 20 percent on 2,000 questions narrows to about 18 to 22.

The interval covers sampling luck and nothing else, and sampling is the smallest of the three problems. Run the composite’s numbers: a system that erred on 38 percent of answers would show six or fewer bad ones in 30 only about three times in a hundred. A jump that size points to the other two problems. The questions a team types rarely match the mix users send, and when nobody has written down what counts as a hallucination, each tester applies a different standard.

Which numbers should a RAG team report?

One rate hides where the failure is. TruEra’s RAG triad, used in the open-source TruLens library, splits it along the pipeline into three checks, and each one points at a different repair. Ragas scores groundedness the way I would want an auditor to: it breaks the answer into short statements and divides the number supported by the retrieved passages by the total.

I add a fourth number: how often the system declines to answer. In the Stanford study, Ask Practical Law AI gave incomplete answers, refusals or responses with no grounding, to 63 percent of questions. A system can lower its hallucination rate by refusing, so report both. A refusal is cheaper than a wrong answer, but the user still leaves without one.

The last row matters because the triad has a ceiling. TruLens says a system that passes all three checks is free of hallucination “up to the limit of its knowledge base”. If the policy document is out of date, a perfectly grounded answer is still wrong. Only a reference answer written by someone who knows the domain catches that.

What will the numbers tell you to fix?

Measured numbers usually point somewhere other than a bigger model. Three repairs come up most often.

1. Retrieval first. Combining keyword search with embeddings and then re-ranking the results is the usual fix for low context relevance. In Anthropic’s published tests in September 2024, adding context to chunks and pairing embeddings with keyword search cut the share of relevant passages missed from the top 20 from 5.7 to 2.9 percent, and a re-ranker took it to 1.9. Those were Anthropic’s datasets; measure yours.
2. Less context, better placed. Liu and colleagues found that models use information at the start or end of a long context better than information in the middle.
3. Tell the model what it does not know. When retrieval comes back thin, say so in the prompt and let the system answer “I do not have enough information”. Cite the passage, not just the document, so a reviewer can check the claim in seconds. Then watch the refusal rate, because this repair moves errors into refusals.

How do you build the test set nobody builds?

Build it before the pipeline. A team that writes the test set first argues about what “correct” means while that argument is still cheap.

1. Real questions. Take them from support tickets, search logs and, once it runs, the pilot’s transcripts. Questions the team invents drift toward the cases it already thought of.
2. The hard cases on purpose. Include questions about dates and versions, questions that span two documents, questions with a false premise, and questions the corpus cannot answer.
3. A written scoring rule. Decide before scoring what counts as a hallucination, and whether you count answers or claims. The Stanford rule is a good start: false, or cited to a source that does not support it.
4. A judge you have checked. An LLM judge can do the volume. Zheng and colleagues found GPT-4 agreed with human preferences between chat answers over 80 percent of the time, about as often as humans agreed with each other. Checking claims against a source is a harder task, and ARES uses a few hundred human labels to correct its judge’s estimate. Label a sample by hand and measure the judge before you trust it.
5. A held-out set. Tune on one set of questions and decide launches on another that nobody tunes against.
6. Enough of them, and a comparison. Several hundred labelled questions is a sensible floor for a launch decision: at 300, a 20 percent rate is pinned to within about five points. Score today’s process, a person with the intranet search, on the same set, report each hard-case group on its own, and add the questions production gets wrong.

For some systems this stops being good practice and becomes law. Under Article 15(3) of the EU AI Act, the accuracy levels and accuracy metrics of a high-risk AI system must be declared in its instructions for use. For the uses listed in Annex III, such as hiring and credit decisions, that applies from 2 December 2027, the date set by the Digital Omnibus. A guess can be declared once. It cannot be defended.

What should you ask on Monday?

The next time someone quotes a hallucination rate in a steering committee, ask three questions.

1. How many questions is it based on, and where did they come from?
2. What counted as a hallucination, and who decided?
3. What is the margin of error, and when was it last measured?

If the answers are “about thirty”, “we looked at them” and “not sure”, the number is a guess. Say so, and fund the test set before the next feature. Boards face the same problem one level up, and I describe it in The Evaluation Gap: Why Good Boards Fund Bad AI. A team starting from little can put the test set inside minimum viable governance from the first week.

Sources

• Varun Magesh, Faiz Surani, Matthew Dahl, Mirac Suzgun, Christopher D. Manning and Daniel E. Ho, "Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools", arXiv, 30 May 2024; Journal of Empirical Legal Studies 22(2), 216 to 242, 2025.
• TruLens, "The RAG Triad", documentation, read 6 October 2026.
• Shahul Es, Jithin James, Luis Espinosa-Anke and Steven Schockaert, "Ragas: Automated Evaluation of Retrieval Augmented Generation", arXiv, 26 September 2023.
• Lianmin Zheng and others, "Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena", NeurIPS 2023 Datasets and Benchmarks Track, arXiv, 9 June 2023.
• Jon Saad-Falcon, Omar Khattab, Christopher Potts and Matei Zaharia, "ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systems", NAACL 2024, arXiv, 16 November 2023.
• Anthropic, "Introducing Contextual Retrieval", 19 September 2024.
• Nelson F. Liu and others, "Lost in the Middle: How Language Models Use Long Contexts", Transactions of the Association for Computational Linguistics, arXiv, 6 July 2023.
• Edwin B. Wilson, "Probable Inference, the Law of Succession, and Statistical Inference", Journal of the American Statistical Association 22(158), 209 to 212, June 1927.
• Regulation (EU) 2024/1689 (the AI Act), Article 15 and Annex III, as amended by Regulation (EU) 2026/1744, Official Journal, 24 July 2026.

All sources checked on 6 October 2026.

Changes on 6 October 2026. Rewritten from about 2,800 words as an Argument with a new headline, and merged with “The RAG Renaissance: Engineering Trust Into AI Systems”, whose address now leads here. The opening case is now labelled as a composite with illustrative figures. The Stanford study, the margin-of-error figures and the retrieval test results are new. Removed: the unsourced claim of reviews at more than thirty enterprises, client anecdotes with figures that could not be documented, enterprise target thresholds with no source, a vendor ranking, a 12-week roadmap, and six interactive blocks.