Found but Not Read

Abstract

Evidence found but not used

Retrieval-augmented document question answering assumes that once the right page is found, a vision–language model (VLM) can read it. We show that this assumption often fails, leaving a retrieval–reading gap: evidence found but not used.

A paired protocol isolates this gap by comparing answers from the retrieved page images alone with answers from the same images plus their extracted text. On FoveDoc-Bench, our benchmark with traceable evidence, retrieval finds nearly every evidence page, yet adding CPU-OCR text raises strict accuracy by 13 to 16 points. An exact text layer roughly doubles the gain, which appears across six VLMs from three families and, within one family, narrows with scale without closing. The reader can read this evidence but cannot find it: crops of it recover most of the text gain, boxes around it on the page none.

The same protocol identifies two boundaries. Extracted text helps on textual evidence but is neutral or harmful on charts and figures. Its advantage shrinks as retrieval degrades, and unrelated text of the same form adds nothing detectable. Extracted text is an amplifier of retrieval that works, not a substitute for retrieval that does not.

The retrieval–reading gap

Reading is not the easy part

The protocol holds retrieval at saturation, so nearly every evidence page is in front of the reader (full recall@16 is 1.000 on V-NIAH and 0.998 on V-MQAR), and asks the same question twice: once from the 16 retrieved page images, once from the same images plus a text memory extracted from them: the 16 text blocks that score highest lexically against the question, at most 300 characters each. Nothing is removed, so any improvement is evidence the reader held and did not use.

Strict accuracy, stock Qwen3-VL-2B-Instruct, n = 1,173 per task, paired on identical pages. OCR is RapidOCR (PP-OCR models) on CPU over the rendered pages; the PDF text layer is the exact born-digital text. OCR contrasts: 95% CIs [+0.109, +0.158] and [+0.127, +0.187], both p < 10⁻²³.

What OCR gives up is fidelity, not volume. At the fixed block budget the OCR arm packs more characters than the text-layer arm on 77% of samples. But the best packable OCR block recovers 76% of a gold excerpt's tokens on average: the share of the text-layer gain that OCR keeps tracks the share of gold evidence that OCR returns intact.

Across backbones

The gap is not a property of one backbone

With the text-layer memory, the gain appears in all six stock readers we test: Qwen3-VL-2B/4B/8B, Qwen2.5-VL-7B, LLaVA-OneVision-7B and Idefics3-8B. Within Qwen3-VL, at identical settings, it narrows monotonically with scale, from +0.28 to +0.23 on V-MQAR and from +0.36 to +0.23 on V-NIAH between 2B and 8B. A stronger visual reader needs the text less, and at 8B still needs it.

Cheaper levers do not close it. With a 4B reader on V-MQAR, doubling the block budget, a union selector and a reasoning-mode reader each move accuracy by at most +0.03; self-ask decomposition costs −0.043; a 2B LoRA recovers +0.06 on V-MQAR and +0.05 on V-NIAH, and a 4B LoRA adds +0.009.

Evidence accessibility

The reader can read the evidence but cannot find it

Three paired controls locate the failure. The 16 page images stay fixed in every arm; only what is added to them changes.

Paired gain in strict accuracy over the images arm, stock Qwen3-VL-2B, n = 1,173 per task. Percentages are the share of the 16-block text gain the crops recover. Crops are cut at the reader's own page pixel density.

The crops carry pixels the pages already held. The reader can read the evidence but cannot find it among 16 dense pages; lifting it out, as text or as a crop, makes it usable. Pointing at it on the page does not.

Page-level retrieval is not evidence-level retrieval. With every gold page in view, the reader still leaves 13 to 16 points of evidence unused in every backbone we test. Lifting the evidence off the page recovers it, with or without question-conditioned selection.

Boundary 1

The benefit follows the evidence modality

MMLongBench-Doc differs in the two respects that matter: its evidence is mostly graphical, and its retrieval is not saturated (full recall@16 is 0.844). On the 414 questions with exactly one evidence tag, the paired effect is +0.085 on text (n = 129) against −0.046 on charts or figures (n = 285), a difference of +0.131 [+0.046, +0.215] (p = 0.0024). A flat transcript discards the structure a chart or figure encodes.

Paired strict accuracy; same reader, retriever, memory builder and judge as on FoveDoc-Bench. Bonferroni α = 0.0125 over the four strata.

Chart and figure evidence weighs 55% of the reading questions against 23% text-only, so the two effects cancel in aggregate: the pooled advantage is +0.004. On unanswerable questions the memory arm more often reports that the text contains no answer, where the image arm makes one up.

A modality boundary. Extracted text helps textual evidence and hurts graphical evidence. An aggregate over a mixed benchmark hides the 13.1-point contrast.

Boundary 2

Retrieved evidence buys most of the advantage

Retrieval on FoveDoc-Bench is saturated, so we degrade it by hand. FoveDoc-Bench's evidence annotations make three controls possible:

1. Control 1 withdraws gold pages and substitutes pages ranked below the top 16, so recall moves and the page budget does not.
2. Control 2 drops zero-recall samples whose packed memory still contains the answer string, so the residual is not a leaked answer.
3. Control 3 replaces each document's memory with another document's text under the same packing rule, so only whose text it is changes.

The advantage falls from +0.125 through +0.066 at half recall to +0.030 on V-MQAR, and from +0.149 to +0.043 on V-NIAH. The within-sample change is +0.095 and +0.106 (both p < 10⁻⁹): roughly three quarters of the full-recall advantage is bought by the gold pages being present. On leak-free samples the zero-recall residual is +0.028 on both tasks, and own − foreign memory accounts for all of it. Even at full recall, foreign text is inert rather than harmful; what hurts the reader is relevant text in the wrong modality, not irrelevant text.

A retrieval boundary. Three quarters of the advantage is bought by retrieved evidence and the rest by the document's own context. Unrelated text of the same form adds nothing detectable.

Resource

FoveDoc-Bench

These interventions need evidence that can be traced, withdrawn and re-presented. FoveDoc-Bench has 2,346 questions over 1,173 born-digital documents: annual and technical reports, manuals, academic papers and illustrated technical blogs. Each question carries its answer, a 16–24-page context numbered as in the source, and the evidence pages, blocks and excerpts behind it. V-NIAH asks for one evidence block on one page; V-MQAR joins two blocks on distinct pages. All 1,173 documents are held out from the auxiliary training and validation pools.

Recall@16 is the fraction of questions whose every gold page is among the 16 pages ColQwen2 retrieves.

One question

{
  "id": "gemini32_vmqar_resume_v2_20260314:vmqar:<document_id>:6-29",
  "benchmark": "V-MQAR",
  "question": "Based on the provided mathematical derivations, what is the upper bound for $1 + \\|g_t\\|^2$ and …",
  "answers": ["The upper bound for $1 + \\|g_t\\|^2$ is $1 + \\tau_t$, and …"],
  "images": ["render/<document_id>/page_0006.png", "…", "render/<document_id>/page_0029.png"],
  "meta": {"evidence": [
    {"page_id": 16, "block_id": "p16_b24", "text_excerpt": "Using the definition of α t , we derive: …"},
    {"page_id": 20, "block_id": "p20_b14", "text_excerpt": "E[f (x T ) − f ∗ ] = E[ f ˆ T ] = …"}
  ]}
}

Conclusion

Two practical conclusions

Citation

BibTeX

@misc{xia2026foundbutnotread,
  title         = {Found but Not Read: When Extracted Text Closes the Retrieval--Reading Gap
                   in Document Vision--Language Models},
  author        = {Xia, Qingtao and Cheng, Siyao and Bao, Jiahua and Du, Jiaxing and Liu, Jie},
  year          = {2026},
  eprint        = {2610.02880},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  url           = {https://arxiv.org/abs/2610.02880}
}