Every article about context recall says the same useless thing: “it depends on your use case.”
That is technically true. It is also completely unhelpful if you are trying to decide whether to ship your RAG pipeline or not.
This post gives you actual numbers — the thresholds that matter, what each score range means in practice, what happens to your users when your score is too low, and exactly what to do when it is. We will also cover the most common questions teams have when they first start measuring this metric.
No hedging. No “it depends.” Just the numbers.
First: What Is Context Recall?
Before you can decide if your LLM score is good, you need to understand what context recall actually measures.
When a user asks your RAG system a question, two things happen:
Step 1 — Retrieval: Your system searches through your documents and pulls out the chunks it thinks are most relevant to the question.
Step 2 — Generation: The LLM reads those chunks and writes an answer.
Context recall measures Step 1 only. It asks a specific question: out of all the information needed to answer this correctly, how much of it did the retriever actually find?
The formula, is simple:
Context Recall = (Information Retrieved That Was Needed) ÷ (Total Information Needed)
So if a question requires five pieces of information to answer correctly, and your retriever finds four of them, your context recall is 0.80 — or 80%.
And if the fifth piece is missing, and LLM does not have it. Then we need to understand: the LLM does not know it is missing.
It will write an answer anyway — using whatever data it has at the moment, plus anything from its training that seems relevant. This is the part where hallucinations come from.
A good example:
Assume you have a electronic store office and you integrate a chatbot to get more sales.
So if a user asks your company’s support chatbot: “What is the return policy for electronics bought during the holiday sale?”
To answer this correctly, the retriever needs to find:
• The general return policy document
• The holiday sale terms and conditions
• The electronics-specific exceptions clause
If it only retrieves the first two documents and misses the exceptions clause, your context recall is 0.67. The LLM will write a good answer about electronics returns — but it will not know about the exception, so it will either make something up or give the wrong policy.
The customer will have to act on the incomplete information. You have a problem.
That is what a low context recall score actually means in production.
The Actual Scale to Measure and What the Numbers Mean
Here is the table nobody else will show you — based on production data, not academic benchmarks:
The most important number: 0.82 is the minimum score to ship to external users.
Save this table for next time
Most tutorials and vendor documentation will tell you to aim for 0.9 before shipping. That number comes from academic benchmark datasets, which are significantly cleaner and easier than real production traffic.
In 2026, less than 18% of public RAG pipelines actually hit 0.9 in production, according to data we have collected from teams running context recall testing across hundreds of deployments.
The 0.82 threshold is where user satisfaction actually changes. Below it, wrong answers happen often enough that users lose trust in the system. Above it, errors are infrequent enough that most users never notice.
Why Low Context Recall Causes Hallucinations
Teams often focus on faithfulness — whether the LLM’s answer is consistent with the retrieved context. They get a faithfulness score of 0.91 and feel good about the system. Then they ship it, and users start complaining about wrong answers. They check faithfulness again: still 0.91.
What is going on? This is the part most people get backwards.
The problem is upstream. The retriever missed something important, so the LLM answered faithfully from incomplete context. The answer is consistent with what was retrieved — but what was retrieved was not the full picture.
A real example from a production legal RAG system: the team wanted to go live with 0.91 faithfulness rating on their offline eval set. Three weeks later, users were reporting that 1 in 6 answers missed a key element.
The team checked faithfulness: still 0.91.
They checked context recall: 0.62. The retriever was missing the second statute on multi-hop questions — questions that required combining information from two separate documents. The LLM was answering honestly from the partial context it received, so faithfulness stayed high. But users were getting incomplete legal advice.
Retrieval-augmented legal research tools have showed hallucination rates up to 33%, opposite to what the vendor claims. This happens precisely because of the context recall problem: retrieved passages are quite relevant but factually not quite sufficient to support a complete answer.
Hybrid approaches combining RAG architectures with a strong validation protocol can reduce hallucinations by 54–68% across domains. The “strong validation” part is the key phrase — meaning actually measuring context recall and fixing it before shipping, not after.
The mechanism in simple terms:
Low context recall → LLM receives incomplete information → LLM fills the gaps from training data → training data contains general knowledge, not your specific policies/documents → hallucination.
A faithfulness score alone will not catch this. Context recall is the metric that lives upstream and catches retrieval failures before they become generation failures.
The minimum recommended metric combination is context recall plus faithfulness. The full recommended set is: context relevance, context recall, faithfulness, groundedness, and answer relevance.
Different Use Cases, Different Minimums
There are three tiers of minimum acceptable context recall score, depending on what your RAG system does:
| Use Case | Minimum Context Recall |
|---|---|
| Internal employee chatbot | 0.75 |
| Public customer support chatbot | 0.82 |
| Legal, medical, or financial RAG | 0.90 |
The reason the last category is higher is clear: the cost of a wrong answer is much higher. A wrong answer on an internal HR chatbot is an inconvenience. A wrong answer on a legal research tool can cause a client to miss a filing deadline. A wrong answer on a medical information system can cause patient harm.
A 2025 study from Stanford Law School evaluated leading legal AI tools on 200 open-ended legal research queries. The tools with retrieval recall below 0.85 produced materially incorrect answers on multi-step legal questions at a rate that would not be acceptable in a law firm context.
For public-facing consumer products, the Air Canada case in 2024 is the clearest illustration of why the 0.82 threshold exists. The chatbot gave a passenger incorrect information about bereavement fares after his grandmother died. He booked full-fare flights on that promise. The airline refused to honour it. The tribunal ruled that the company was responsible for what its chatbot said. The ruling established that companies are liable for what their AI systems say — there is no “the chatbot said it, not us” defence.
That is the real-world consequence of shipping a RAG system with context recall too low to reliably find the right policy information.
![ARTICLE[1]](https://qyrus.com/qapi/wp-content/uploads/2026/07/ARTICLE1-1024x523.png)
Context Recall vs Context Precision: What Is the Difference?
These two metrics measure different failure modes, and you need both.
Context recall (what we have been discussing): measures completeness. Did the retriever find all the information it needed? A low recall score means important information is missing.
Context precision measures relevance. Of everything the retriever pulled in, how much of it was actually useful? A low precision score means the retriever is pulling in a lot of noise — documents that are vaguely related but do not help answer the question. This makes the LLM’s job harder because it has to sort through irrelevant information to find the signal.
The interaction between them:
• High recall, low precision: You retrieved everything needed, plus a lot of junk. The LLM may get confused by the noise and still produce a wrong answer, even though the right information was technically in the context. Also increases latency and cost.
• High precision, low recall: Everything you retrieved was relevant, but you missed some important documents. The LLM gives a precise but incomplete answer. For complex questions, this is where hallucinations happen.
• High recall, high precision: The ideal state. You found everything needed, and nothing irrelevant. This is what a well-tuned RAG pipeline looks like.
Production targets for a well-tuned RAG system in 2026: faithfulness ≥0.9, answer relevancy ≥0.85, context precision ≥0.8. Context recall should be at least 0.82 for public-facing systems and 0.90 for high-stakes domains.
Think of it this way: precision is about quality of what you retrieved, recall is about completeness of what you retrieved. A doctor doing a diagnosis wants recall (do not miss anything important).
A spam filter wants precision (do not flag legitimate emails). Most RAG systems need a balance, but for factual question-answering, recall matters more — a missing answer is worse than a noisy one.
How to Actually Measure Context Recall
You cannot improve what you do not measure, and you cannot measure context recall without a proper evaluation setup. Here is what that looks like practically.
What you need:
- A set of test questions (minimum 200 — fewer than this gives you statistically unreliable scores)
- Gold answers for each question (what the correct answer actually is)
- Gold documents for each question (which documents contain the information needed to answer)
- A framework to compute the metric
The Ragas approach (Python, open source):
from ragas import evaluate
from ragas.metrics import context_recall, faithfulness, context_precision
from datasets import Dataset
# Your test data
data = {
“question”: [“What is the return policy for electronics?”, …],
“answer”: [“Electronics can be returned within 30 days…”, …], # LLM output
“contexts”: [[“chunk1 text”, “chunk2 text”], …], # Retrieved chunks
“ground_truth”: [“The correct answer is…”, …] # Your gold answer
}
dataset = Dataset.from_dict(data)
result = evaluate(dataset, metrics=[context_recall, faithfulness, context_precision])
print(result)
# {‘context_recall’: 0.84, ‘faithfulness’: 0.91, ‘context_precision’: 0.78}
Run the metrics offline on every prompt or model change. Block ship on a composite threshold.
What 200 test cases gives you: a margin of error of roughly ±5 percentage points at 95% confidence. A single test run of 20 cases can give you a score that varies by 15–20 points from run to run. You need volume to trust the number.
Common mistake: teams run their eval on the same documents they used to build the RAG system. This inflates scores because the retriever has effectively seen the test data. Build your eval set from documents that were part of the corpus but not used to tune any retrieval parameters.
Track these metrics over time. When you change chunking strategy, embedding model, or retrieval pipeline, you will know immediately if it helped.
My Score Is Too Low. What Do I Fix First?
If your context recall is below 0.82, fix things in this order. Most teams try chunking strategy first because it feels technical and deliberate. That is the wrong order.
Step 1: Increase chunk size by 50%
The most common cause of low recall is chunks that are too small. When you split a document into 256-token chunks, the context needed to answer a question often spans multiple chunks — and the retriever may only find one of them. Increasing to 384 or 512 tokens gives each chunk more complete information. This alone fixes recall by 10–15 points for most pipelines.
Step 2: Increase top-k from 3 to 5
If your retriever is currently pulling 3 chunks per query, try 5. You are giving the LLM more surface area to find the relevant information. Yes, this increases token count and cost slightly — but a hallucination costs more than a few extra tokens.
Step 3: Add a reranker
After your initial retrieval, a reranker scores each chunk against the specific query and reorders them. This significantly improves the quality of what reaches the LLM. Cross-encoder rerankers (like Cohere Rerank or BGE-Reranker) consistently improve both recall and precision. This is the step most teams skip because it adds complexity — but it is often the highest ROI improvement after chunk size and top-k.
Step 4: Change your embedding model
Not all embedding models are equally good at your domain. A general-purpose embedding model will underperform on legal, medical, or technical content compared to a domain-specific model. If steps 1–3 do not get you to 0.82, the embedding model is likely the bottleneck.
Step 5: Revisit chunking strategy
Only after trying the above should you experiment with chunking strategy — semantic chunking, sentence-window chunking, or parent-document retrieval. These are high-effort, high-variance changes. The first four steps are lower effort and higher certainty.
90% of teams that are below 0.82 will get above it with steps 1 and 2 alone.
The Questions Teams Ask Most
“My context recall is 1.0. Is that good?”
No. A perfect score of 1.0 means your test set is broken. Either you have too few test cases, the questions are too simple, or your evaluation questions were built from the same documents you used to tune the retriever. In real-world production with diverse queries, you will not retrieve 100% of needed information every time. If you are seeing 1.0, audit your test set.
“Should I prioritise recall or precision?”
For most question-answering RAG systems, recall first. A missed answer is worse than a noisy context. The exception is when you have very long context windows filling up — at that point, precision becomes critical because the LLM cannot process everything effectively.
“My faithfulness is 0.92 but users say the answers are wrong. Why?”
Because faithfulness and context recall measure different failure modes. Faithfulness tells you whether the LLM’s answer is consistent with what it retrieved. It does not tell you whether what it retrieved was the right information. Check your context recall — it is almost certainly below 0.82.
“Does a good context recall score mean no hallucinations?”
No. Context recall is a necessary condition, not a sufficient one. You can have 0.95 recall and still have hallucinations — the LLM can fabricate even when the right context is present, especially on long contexts where important information gets “lost in the middle.” You need both high recall and high faithfulness, plus evaluation that runs continuously in production, not just before launch.
“How often should I run context recall evaluation?”
On every significant change to the pipeline: every prompt change, every embedding model update, every chunking strategy change, every time you add or update documents in your knowledge base. Quality cited as the top barrier to deployment by 32% of respondents in LangChain’s 2026 State of AI Agents report — continuous evaluation is how you stay below that threshold rather than discovering problems after users do.
Measuring Context Recall at Scale
Running 200 eval cases manually before every deployment is not realistic for most teams. The answer is to automate it as part of your CI/CD pipeline — the same way you run unit tests before merging code.
The setup looks like this: every time a change goes to your RAG pipeline, an automated eval run fires against your 200-case test set. If context recall drops below your threshold (say 0.82 for a customer-facing product), the pipeline fails and the change does not ship. This is how you catch the regression that a prompt tweak introduced before it reaches your users.
Teams that run evaluation this way catch regressions that would otherwise take weeks to surface through user complaints — by which point the reputational and operational damage is already done.
qAPI is built for exactly this workflow: define your test cases, define your thresholds, and run them automatically against your RAG pipeline on every change. The dashboard shows context recall, faithfulness, and precision trends over time — not just a one-time score, but a signal you can act on continuously. You can run a full 200-case evaluation in about two minutes.
The Three Things to Remember
If you read nothing else in this guide, take these three things:
- 0.82 is the minimum for public-facing RAG.Below that, usersencounter wrong answers often enough to lose trust in the system. The “aim for 0.9” advice from most tutorials is based on academic benchmarks, not production data.
- Context recall and faithfulness measure different failures.A high faithfulness score does not mean your retriever is finding everything it needs. Measure both. Diagnose them separately.
- Scores above 0.96 are a red flag, not a win.If you are hitting that number, your test set isprobably not representative of your real production traffic. Make it harder.
Measuring context recall on your own RAG pipeline? qAPI runs a full case evaluation automatically on every deploy — so you know if a change hurt your retrieval before your users do. Start free →
