# Your retriever's recall is not your answer quality.

A team has a RAG system that gives mediocre answers, and a theory about why. The answers are not wrong, exactly — they are thin, they miss the point of the question, a reviewer reading them keeps thinking the system should have known better. The team looks at the pipeline and lands on the retriever as the suspect. It is the plausible suspect: if the answer is weak, the model must not have been handed the right material, and handing the model the right material is the retriever's job. So they instrument the retriever, find that recall@k is lower than they would like, and go to work on it.

The work is real work and it produces real movement. A stronger embedding model goes in. The top-k is raised — top-20 retrieved per query instead of top-5, on the reasoning that more retrieved passages can only mean more chances for the answer-bearing one to be present. A recall@k metric goes on the dashboard and it climbs, week over week, exactly the way a metric you are optimizing is supposed to climb. The retriever, by its own number, is getting better.

The answer quality does not move. The team re-runs the answer eval expecting the retriever gains to have flowed downstream, and the end-to-end number is flat. In some slices it is worse than where it started. The two metrics — the one on the dashboard and the one anybody actually cares about — have come apart, and the team has spent a quarter optimizing the one that was not the problem.

This post is about why retrieval recall@k and final answer quality are decoupled in RAG — why raising recall, and simply stuffing more retrieved context into the prompt, routinely fails to improve and often lowers the answer, and why optimizing the retriever's recall is not the same thing as optimizing the system.

## Recall@k is a component metric, not a system metric

The mistake is a level error, and it is worth naming precisely, because the retriever metric sits so close to the answer that it feels like a proxy for it.

Recall@k measures one component in isolation. It asks: of the documents that should have been retrieved for this query, what fraction landed in the top k? It is computed entirely inside the retrieval stage, against a labeled set of relevant documents, with no model and no generated answer anywhere in the calculation. It is a clean, cheap, well-defined number, and it tells you something true — about the retriever.

Answer quality is a property of the whole pipeline. It is what comes out the far end after the retriever has run, the retrieved passages have been packed into a prompt, and the generation model has read that prompt and written a response. Exact-match, recall of the gold answer string, a faithfulness score, a human judgment — whatever the answer metric is, it is measured on the output of the system, and the system includes everything recall@k does not: the prompt construction, the context length, the model's ability to find the relevant passage among the others, and the model's ability to use it without being pulled off course by the passages around it.

Between the retriever's number and the system's number sits the entire generation stage, and that stage is not a wire. It is a lossy, length-sensitive, distractible component with failure modes of its own. A retriever metric improving is an input to that stage getting better on one axis. Whether the output gets better depends on how the stage responds — and the research that follows shows the stage responds in ways that break the proportionality a team is implicitly assuming. Recall@k is a real diagnostic for the retriever. It was never a measurement of the system, and treating it as one is how a quarter disappears.

## More retrieved context is not more signal

Raising top-k feels monotonic. Each additional retrieved passage is one more passage in the prompt, and one more passage is, the intuition says, one more chance that the answer is in front of the model — strictly more information, never less.

It is not strictly more information, because the prompt is not a set the model consults at leisure. It is a single context the model reads in one pass, and every passage you add competes for the model's attention with every passage already there. Add a passage that does not bear on the question and you have not added a chance; you have added a distractor — a fluent, on-topic-looking, retrieved-because-it-was-similar block of text that the model now has to read past to find the passage that matters. The answer-bearing passage does not get easier to use as you retrieve more. It gets harder to find, because it is now buried among near-misses that the retriever ranked highly precisely because they resemble it.

So the generation stage faces a trade the retriever metric cannot see. Raising k raises the chance the gold passage is somewhere in the prompt — the thing recall@k rewards — and at the same time raises the count of distractors the model must survive to use it. Past a small number of passages the second effect starts winning. More retrieved context stops being more signal and becomes more noise with a little signal in it, and the answer metric, which measures what the model actually produced, registers the noise. The dashboard sees k go up and recall follow. The answer sees a harder prompt.

## What the research shows

The decoupling is not a hunch, and it is not one lab's artifact. Three studies measure it from three directions — recall moving without the answer moving, documents being added and flipping correct answers to wrong, and raw context length degrading a model that was handed perfect retrieval. Read them in order, because each one closes an escape hatch the previous one leaves open.

Start with the cleanest statement that recall and answer quality are decoupled. [_Toward Optimal Search and Retrieval for RAG_](https://arxiv.org/abs/2411.07396) (arXiv 2411.07396) deliberately degraded the retriever to see what the answer would do. Cutting search recall@10 down to 0.7 — a real, substantial loss of raw search accuracy — produced only a 2–3% drop in gold-document recall and left QA quality essentially unchanged. Sit with the shape of that. A retriever can shed a meaningful amount of its search exhaustiveness and the answers barely notice, because the documents that actually carry the answer were still getting through. The retriever's headline number fell and the system's number did not follow it down.

The same paper makes the decoupling cut the other way, which is the half a team optimizing recall most needs to hear. It added retrieved-but-irrelevant documents — "noisy" passages, the kind a higher top-k drags in — alongside the gold ones, and answer quality consistently degraded. On the ASQA dataset with Mistral, adding highly-similar noise next to 5 gold documents dropped EM Recall from 50.73% to 49.91%. The gold documents were all still present; recall of the answer-bearing material was not the variable. The variable was the noise packed in around it, and the noise cost accuracy. The paper's own conclusion is the line to take away: it is gold-document recall — did the answer-bearing document make it in — and not search exhaustiveness that drives QA performance. Recall is decoupled from quality in both directions: you can lose raw search recall almost for free, and you can hold the gold documents fixed and still lose accuracy by adding more context around them.

The second study isolates the act of adding documents and measures what it does to answers that were already correct. [_Understanding the Design Decisions of Retrieval-Augmented Generation Systems_](https://arxiv.org/abs/2411.19463) (arXiv 2411.19463) ran a "more-is-worse" analysis using k=1 as the baseline and tracked what happened as documents were added past the first. Adding documents flipped previously-correct answers to wrong at error rates of 4.2% to 19.0% — at the top of that range, gpt-3.5-turbo on the CoNaLa dataset had 19.0% of its k=1-correct samples fail once k was raised to 10. These were answers the system got right with a single document, broken by the supposedly safe act of retrieving more. The same paper finds the optimal document count for QA is small — between 5 and 10 — with diminishing returns beyond. The team that raised top-5 to top-20 to "give the model more to work with" was, by this measurement, walking answers off a cliff.

That paper carries one more finding that should unsettle any theory in which the retriever is the whole problem. RAG systems fail on questions the base model answers correctly on its own — and this happens even when perfect documents are supplied. The effect ranges from 2.4% to 12.6% across datasets: on as much as 12.6% of samples, the model knew the answer unaided, was then handed flawless retrieval, and got it wrong anyway, through misinterpretation and improper use of the retrieved knowledge. Perfect retrieval — recall@k at its ceiling — and the answer still regressed. If a flawless retriever can make an answer worse, no amount of retriever optimization is going to close the gap on its own.

The third study removes retrieval from the picture entirely, to prove that part of the damage is not about relevance at all. [_Context Length Alone Hurts LLM Performance Despite Perfect Retrieval_](https://arxiv.org/abs/2510.05381) (arXiv 2510.05381) supplied all the relevant information — perfectly retrieved, fully present in the prompt — and then simply made the input longer. Accuracy degraded by 13.9%–85% as input length grew, all of it within the models' claimed context limits. The natural objection is that longer inputs smuggle in distractor content, so the paper killed that objection: it ran the same test with the filler tokens replaced by blank whitespace and still saw degradation of at least 7% at 30K space tokens, and with the filler masked saw at least 7.9% at 30K masked tokens. There was nothing to be distracted by — the filler was blanks — and the answer still got worse. It is the length itself that costs accuracy. The paper tried a recitation-based mitigation and got only a modest gain — a maximum of 4% at 32K on RULER QA — which tells you the effect is not something a prompt trick dissolves.

Attribute this third result correctly, because it is easy to overclaim. It is not a retrieval finding. It does not say a better retriever hurts. It is the long-context-alone pillar of the argument: even with retrieval held perfect, pure context volume degrades the answer. A team whose instinct is "retrieve more, fill the window" is, independent of recall, paying a length tax this paper measured. And note what none of the three papers claims, because the slug invites the misreading: not one of them shows that higher recall monotonically lowers accuracy. The claim is decoupling — recall and answer quality move on their own clocks — plus a measured cost to adding context, not a simple inverse law.

## Retrieval still matters — gold-document recall is the part that does

The slug of this post overstates by a hair, and the correction is load-bearing enough to be worth its own section.

Read carelessly, the numbers above say retrieval quality does not matter and the retriever is not worth tuning. Read correctly, they do not — and the same papers that supply the warnings say so directly. Retrieval matters. What [_Toward Optimal Search and Retrieval for RAG_](https://arxiv.org/abs/2411.07396) found is not that retrieval is irrelevant — it is that the part of retrieval that drives the answer is gold-document recall, whether the specific answer-bearing document made it into the prompt. That is a retrieval property. A retriever that fails to surface the document containing the answer will produce a worse answer, reliably, and improving that is real system work. The post is not against retrieval. It is against conflating two different things with the answer: raw search recall@k, and sheer context volume.

The distinction is the whole point. Raw search recall@k rewards exhaustiveness — how much of the labeled-relevant set you swept up — and the research shows you can lose a lot of that almost for free. Gold-document recall rewards precision on the thing that matters — did the answer-bearing passage arrive — and that genuinely tracks answer quality. They are not the same metric, and a dashboard that shows recall@k climbing is not necessarily showing gold-document recall climbing; it may be showing you swept up more near-relevant passages, which is the noise the first paper measured a cost for.

Two more limits keep this honest. The optimal document count is small but it is task-dependent — the 5-to-10 range from [_Understanding the Design Decisions of Retrieval-Augmented Generation Systems_](https://arxiv.org/abs/2411.19463) is a finding about the QA settings it tested, not a universal constant; a multi-hop task that genuinely needs to synthesize across many sources will sit at a different point, and you find your point by measuring, not by quoting this one. And the context-length result is a degradation curve, not an on-off switch — short, tight prompts are not magic, they are simply further up a slope that gets steeper as you load the window. The honest claim is narrow and it is this: gold-document recall drives answers and is worth optimizing; raw recall@k and context volume are not reliable proxies for answer quality and should not be optimized as if they were. Measure the answer, not the retriever, as the system's number.

## Fixing the metric

You cannot make recall@k predict answer quality by wanting it to. You can stop letting it stand in for the system metric, and build the evaluation and the pipeline around the number that actually matters.

**Make end-to-end answer quality the system KPI.** The number that gates a launch, sits on the leadership dashboard, and defines whether the project is working is the answer metric — exact-match, gold-answer recall, faithfulness, a calibrated human or judge score — measured on the output of the whole pipeline. Recall@k can sit next to it as a diagnostic, clearly labeled as a retriever-component number. It must not be the headline. A RAG project reporting "recall@k is up" to a decision-maker has reported a component improvement and called it a system result, and the two come apart exactly when it matters. Treat answer quality the way [eval-driven development](/blog/eval-driven-development/) treats the graded set: the eval is the specification, and the eval here is the answer, not the retrieval.

**Keep retrieved context small and high-precision, not large and high-volume.** The literature's optimal range is small — between 5 and 10 documents for the QA settings studied — and adding past it flips correct answers to wrong at measurable rates. Default to a small top-k, treat raising it as a change that must be justified by an end-to-end answer measurement and not by a recall@k gain, and remember the context-length result: even perfectly relevant passages cost accuracy once the window gets long. The goal of the retrieval stage is to put the answer-bearing passage in a short prompt, not to fill the window.

**Spend on precision before you spend on volume — a reranker over a higher k.** If answers are weak because the gold passage is not making it in, the fix is to retrieve the right passage, not more passages. A cross-encoder reranker raises the precision of what reaches the prompt — it can pull the answer-bearing passage to the top and let you keep top-k small — and that routinely beats simply retrieving more. The trade is real and worth reading [reranking is the cheapest RAG win you are not using](/blog/reranking-rag/) for: a reranker that improves precision earns its place where a higher top-k that improves only recall@k does not.

**Diagnose the retriever with gold-document recall, not raw recall@k.** When you do measure the retriever as a component — which you should — measure the variable that the research ties to answers: did the specific answer-bearing document make it into the retrieved set. That is the retriever number worth moving. Raw search recall@k, the fraction of the labeled-relevant set swept up, is the one the research shows you can lose cheaply and gain uselessly.

**When answers are weak, do not assume the retriever is the cause.** The design-decisions paper found RAG failing on up to 12.6% of questions the base model already answered correctly, even with perfect documents supplied — a generation-stage failure, not a retrieval one. Before a quarter goes into the retriever, check whether the gold passage is already arriving and the model is misusing it. If it is, the work is in the prompt, the context size, and the generation step. Whether the failure is retrieval or generation is itself a measurement — the split between [faithfulness and groundedness](/blog/faithfulness-vs-groundedness/) is one way to localize it — and the chunk the passage arrives in is part of the picture too, which is the argument of [you don't have a RAG problem, you have a chunking problem](/blog/chunking-not-rag/).

**Treat the context window as a budget, not a free input.** Every retrieved passage you add is read, costs latency and tokens, and — past the small optimal range — costs accuracy. The discipline that [context engineering for agents](/blog/context-engineering-agents/) applies to an agent's prompt applies to a RAG prompt: what goes in the window is a deliberate, measured decision, and "retrieve more, it can only help" is the assumption the research most directly refutes.

## The checklist

Before you call a RAG system's retrieval "good enough":

- [ ] The system KPI is end-to-end answer quality, measured on the pipeline's output — recall@k is labeled as a retriever-component diagnostic, not the headline number.
- [ ] No retriever improvement is declared a system improvement without an end-to-end answer measurement confirming it flowed downstream.
- [ ] Retrieved context defaults to a small top-k, in the literature's 5-to-10 range, with a higher k justified by an answer measurement on the actual task.
- [ ] Raising top-k is treated as a change that can lower answer quality, not as a safe always-on improvement.
- [ ] The retriever is diagnosed with gold-document recall — did the answer-bearing document arrive — not raw search recall@k.
- [ ] Precision-raising changes, such as a cross-encoder reranker, are tried before volume-raising ones when answers are weak.
- [ ] Weak answers are checked for generation-stage failure — the gold passage arriving and being misused — before the retriever is blamed.
- [ ] The context window is treated as a budget; long prompts of even relevant passages are understood to cost accuracy.

## Reading list

- [_Toward Optimal Search and Retrieval for RAG_](https://arxiv.org/abs/2411.07396) (arXiv 2411.07396) — cutting search recall@10 to 0.7 leaves QA quality essentially unchanged, while adding similar-but-irrelevant noise alongside gold documents drops EM Recall from 50.73% to 49.91%: recall is decoupled from answer quality in both directions, and gold-document recall is what drives QA.
- [_Understanding the Design Decisions of Retrieval-Augmented Generation Systems_](https://arxiv.org/abs/2411.19463) (arXiv 2411.19463) — adding documents past the first flips correct answers to wrong at 4.2%–19.0% error rates, the optimal document count is 5–10, and RAG fails on up to 12.6% of questions the base model answered correctly even with perfect documents supplied.
- [_Context Length Alone Hurts LLM Performance Despite Perfect Retrieval_](https://arxiv.org/abs/2510.05381) (arXiv 2510.05381) — with all relevant information perfectly retrieved, accuracy still degrades 13.9%–85% as input length grows, persisting even with blank or masked filler — it is the length itself, not distractor content.

A retriever metric and an answer metric are both numbers on a dashboard, and at a glance they look like the same story told twice. They are not. One measures a component on one axis; the other measures the system a user actually meets, and the research is consistent that the two move on separate clocks — recall can fall almost for free, more context can make the answer worse, and pure length degrades a model you fed perfect retrieval. Optimize the answer. The retriever's number is a diagnostic you read on the way there, not the destination.