Best AI Hallucination Detection Tools for QA Teams
Compare Galileo, Arize Phoenix, DeepEval and Ragas for catching AI hallucinations, see why Patronus Lynx is retired, and try a runnable grounding check.

Which AI hallucination detection tool should you use?
For most QA teams, the practical shortlist is Galileo, Arize Phoenix, DeepEval and Ragas, with a few simple rule-based checks of your own underneath. Choose Galileo if you want a managed platform with a ready-made Context Adherence metric and an option to run checks on cheaper small models at scale. Choose Arize Phoenix if you want open-source tracing plus separate Faithfulness and Hallucination evaluators you can run on recorded conversations. Choose DeepEval if you want hallucination checks written as Python tests that fail a CI build. Choose Ragas if your feature is a retrieval-based assistant and you mainly need a faithfulness score.
Patronus AI, often named for its Lynx hallucination model, is no longer an option for new projects: its documentation site states that the evaluation platform, API, SDKs and Lynx are retired and new users cannot sign up. The Lynx research paper stays public.
This comparison uses each vendor's official documentation, checked in November 2026. We did not run a hands-on benchmark of the tools. The Python example later in the article is an illustrative scenario that we did run locally.
What a hallucination is, in testing terms
A hallucination is an answer that states something the AI had no basis for. For QA work it helps to split it into two types, because tools detect them differently:
- Closed-domain (grounding) hallucination: the answer adds or changes facts that are not in the material the AI was given, such as a policy document, a search result or a tool response. This is the type most tools measure, and it is the one you can test reliably.
- Open-domain (world-knowledge) hallucination: the answer is wrong about the real world with no supplied source. Detecting this needs a trusted reference answer or a human, because there is nothing in the request to compare against.
Most products QA teams test today are grounded: a support bot that reads help articles, an HR assistant that reads policies, an agent that calls an order API. So the core question becomes simple and testable: is every claim in the answer supported by what the AI saw?
Two terms appear in almost every tool. Faithfulness (or groundedness) scores how well an answer sticks to the supplied context. An LLM-as-a-judge is a second AI model that reads the question, context and answer and decides whether the answer is supported. Most hallucination scores are produced by a judge, which is why the judge itself needs testing.
How we compared the tools
We used criteria a tester can check during a short pilot, not marketing claims:
- What the score is grounded against. A single block of retrieved text, the whole conversation including tool results, or a reference answer.
- Where it runs. Inside your test suite, on recorded production traces, or both.
- Explanations. Does a failing score tell you which claim was unsupported, so a developer can fix it?
- Cost at volume. Judge-based checks call a model for every answer, so running them on all traffic can be expensive. We note documented options for cheaper evaluation without quoting prices.
- Open source or managed. Self-hosting matters when test data cannot leave your network.
- CI fit. Can a hallucination regression fail a pull request?
AI hallucination detection tools at a glance
ACCELQ is listed first as an adjacent functional test automation platform and a past SoftwareTestPilot partner. We do not claim it scores hallucinations; it is relevant because the screens and APIs around an AI feature still need normal regression coverage. The comparison is based on the official documentation linked in each section below.
| Tool | What it checks | Grounded against | Best fit |
|---|---|---|---|
| ACCELQ | Functional web, API and mobile flows | Expected application behaviour | Regression of the product around the AI feature |
| Galileo | Context Adherence (closed-domain hallucination), Completeness | Retrieved context | Managed evaluation and monitoring at scale |
| Arize Phoenix | Faithfulness and conversation-level Hallucination evaluators | Context block, or whole conversation with tool results | Open-source tracing plus evaluation of agents |
| DeepEval | HallucinationMetric, FaithfulnessMetric | Supplied context or retrieval context | Python tests that fail CI builds |
| Ragas | Faithfulness for retrieval-based answers | Retrieved context | Scoring RAG pipelines in notebooks or scripts |
| Patronus AI (Lynx) | Retired as a product | Not applicable | Read the paper; do not plan new work on it |
Galileo: managed Context Adherence with a small-model option
Galileo's Context Adherence metric is defined as a measure of closed-domain hallucinations: cases where the model said things that were not in the context it was given. A score near 1 means the answer only uses supplied information; near 0 means it probably contains facts from elsewhere. The metric sits alongside other RAG metrics such as Context Relevance and Completeness, which helps you separate "the search found the wrong document" from "the model ignored the right document".
For volume, Galileo documents Luna-2, a family of small evaluation models meant to cut the cost and delay of judge calls. The docs state Luna-2 is only available on the Enterprise tier, so confirm access before you design around it.
Strengths: ready-made metric with a clear definition, retrieval and generation metrics side by side, a route to cheaper evaluation of large traffic.
Limitations: it is a managed platform, so check where your data is processed. Galileo's published accuracy figures come from its own internal dataset, so measure agreement on your own labelled examples.
Choose it when you want a managed product that covers both pre-release tests and production monitoring.
Arize Phoenix: open-source, with a conversation-level evaluator
Phoenix is Arize's open-source tracing and evaluation tool. Its Hallucination evaluator checks whether an assistant response contains claims unsupported by, or contradicting, the conversation it had access to: earlier turns, tool calls, tool results and retrieved context. The docs position it as the conversation-level counterpart to the Faithfulness evaluator, which grounds an answer in a single block of retrieved text.
That distinction matters for agents. A common agent failure is stating data a tool never returned, or returned as an error. A single-context faithfulness check can miss this; a conversation-level check is designed for it. According to the source code, the evaluator returns a label (grounded or hallucinated), a score and an explanation from the judge model, and needs a model that supports tool calling or structured output.
Strengths: open source and self-hostable, works on traces you already record, two evaluators for two grounding scopes.
Limitations: you choose and pay for the judge model, and you own setup and upkeep if you self-host.
Choose it when you test agents or multi-turn assistants and want to keep data on your own servers.
DeepEval: hallucination checks as Python tests
DeepEval, from Confident AI, treats evaluations like unit tests. Its HallucinationMetric compares the actual output against a supplied context list using a judge model, and its FaithfulnessMetric compares output against retrieval_context for RAG pipelines. The docs recommend Faithfulness, not Hallucination, for RAG systems, which is a useful hint: pick the metric that matches where your context comes from.
Each metric has a threshold, and a test case fails when the score falls below it, so a hallucination regression can stop a build like any failing test. Metrics also return a written reason, which makes failures easier to triage.
Strengths: familiar test-runner workflow, open source, easy to add to an existing Python suite.
Limitations: judge calls add cost and run time to CI, so keep the gated set small and stable.
Choose it when your team already writes Python tests and wants hallucination checks in the same pipeline.
Ragas: a focused faithfulness score for RAG
Ragas is an open-source library for evaluating retrieval-augmented generation. Its Faithfulness metric breaks an answer into individual claims and checks how many of them can be inferred from the retrieved context. The result is the share of supported claims, which is easy to explain to stakeholders: "8 of 10 claims were backed by the documents".
Strengths: claim-level scoring, light to add to scripts or notebooks, pairs well with retrieval metrics.
Limitations: it is a library, not a monitoring product; you build reporting and alerting yourself.
Choose it when you mainly test a document-based assistant and want a transparent faithfulness number.
For a deeper look at retrieval scoring, see our guide to the best RAG evaluation tools.
What happened to Patronus AI and Lynx?
Many older lists recommend Patronus AI and its Lynx model for hallucination detection. Lynx was published as an open-source model for detecting hallucinations in RAG answers, evaluated on a benchmark called HaluBench. Today Patronus's Lynx page says Lynx is retired as a product and the platform is no longer offered; the research paper remains public.
If an inherited test plan depends on Patronus, plan a migration: export your test cases and labelled examples, pick one of the tools above, and re-measure agreement between the new judge and your human labels before trusting its scores.
Worked example: catching an invented refund window
This is an illustrative scenario. A shop's support assistant answers refund questions from one policy snippet. Before spending money on judge calls, a tester adds a cheap rule: any number in the answer must also appear in the context. Numbers such as days, amounts and limits are where hallucinations hurt most.
import re
def numbers(text):
return set(re.findall(r"\d+(?:\.\d+)?", text))
def unsupported_numbers(answer, context):
return sorted(numbers(answer) - numbers(context))
context = "Refunds are accepted within 30 days of delivery. Electronics: 15 days. Shipping fee of 99 rupees is not refunded."
cases = [
("What is the refund window?", "You can request a refund within 30 days of delivery."),
("Electronics?", "Electronics can be returned within 90 days."),
("Is shipping refunded?", "No, the 99 rupee shipping fee is not refunded."),
("Can I return after 20 days?", "Yes, all items are returnable for 45 days."),
]
for q, a in cases:
bad = unsupported_numbers(a, context)
print("FLAG" if bad else "ok ", q, bad)
We ran this script locally with Python 3. Output:
ok What is the refund window? []
FLAG Electronics? ['90']
ok Is shipping refunded? []
FLAG Can I return after 20 days? ['45']Two answers invented a number: 90 days for electronics (the policy says 15) and 45 days overall (the policy says 30). Both are caught without any AI judge.
Why it works: numeric claims are easy to compare exactly, and a mismatch is almost always a real defect. Where it stops working: an answer like "Electronics cannot be returned" contains no number, so this rule passes it, even though it contradicts the policy. That is the gap a judge-based metric such as Context Adherence, Faithfulness or Phoenix's Hallucination evaluator fills. Use the rule as a fast first gate and the judge as the second layer.
Practice exercise: add a fifth case, "Shipping fees are fully refunded", and predict the result before running it. Answer: the rule prints ok because there is no number. Write it down as a case your judge-based check must catch.
A practical hallucination testing process
- Collect real questions. Use support tickets and search logs, not only invented prompts. Include questions whose answer is not in the documents; the correct response is "I don't know".
- Label 50 to 100 answers by hand. Mark each as supported or not. This set is how you test the tool.
- Measure judge agreement. Run your chosen metric on the labelled set and count where it disagrees with people. Tune the threshold or judge prompt before trusting it.
- Add rule-based gates. Numbers, dates, product names and links can often be checked exactly, as in the example.
- Gate a small set in CI. Keep 20 to 50 stable cases that fail the build; run the larger set nightly.
- Sample production. Score a share of live traffic and review flagged answers weekly.
- Turn every confirmed hallucination into a regression case. The bug list becomes your test suite.
Common mistakes
- Blaming the model when retrieval failed. If the right document was never retrieved, a faithful answer is impossible. Check retrieval metrics first.
- Trusting the judge blindly. A judge model can also be wrong. Always sample its verdicts against human labels.
- Using the wrong grounding scope. Single-context faithfulness misses invented tool results in agents.
- Only testing answerable questions. Refusing correctly is part of the requirement.
- Reading one average score. Look at the failing examples; an average can hide a dangerous pattern such as wrong prices.
- Running a judge on every CI run with a huge set. It slows builds and adds cost; split gated and nightly sets.
Next step
Hallucination testing builds on core QA skills: test design, negative cases and clear bug reports. Strengthen those with our free software testing courses with certificates, then explore automation and AI testing courses. To compare tools across the wider evaluation picture, read our guides to the best LLM observability platforms and AI agent testing platforms, or the AI red teaming tools comparison for the security side.
Have you caught an AI feature inventing facts during testing? Share how you found it, without sensitive details, in the SoftwareTestPilot QA Community.
References
- 1Patronus AI documentation status— docs.patronus.ai
- 2Patronus Lynx research page— docs.patronus.ai
- 3Galileo Context Adherence— docs.galileo.ai
- 4Galileo RAG metrics overview— docs.galileo.ai
- 5Galileo Luna-2 overview— docs.galileo.ai
- 6Arize Phoenix Hallucination evaluator— arize.com
- 7Phoenix HallucinationEvaluator source— github.com
- 8DeepEval Hallucination metric— deepeval.com
- 9DeepEval Faithfulness metric— deepeval.com
- 10Ragas Faithfulness metric— docs.ragas.io
