Metraipolis › archive › dc_7ae26520315f
From Pixels to Pairs: A Comprehensive Benchmark of LLM-Driven Key-Value Extraction in Noisy Document Settings — arXiv:2609.17538
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.17538
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T13:49:03.423Z
- SHA-256 of the archived text
3bb32bb0d21239162125a8f984711b12b6b578b768874c92744235bf13a0e87c- Stable URL of this page
https://metraipolis.com/documento/dc_7ae26520315f
How to cite
From Pixels to Pairs: A Comprehensive Benchmark of LLM-Driven Key-Value Extraction in Noisy Document Settings — arXiv:2609.17538 — arXiv (en), https://arxiv.org/abs/2609.17538 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7ae26520315f (Metraipolis archive dc_7ae26520315f, sha256 3bb32bb0d21239162125a8f984711b12b6b578b768874c92744235bf13a0e87c)
BibTeX
@misc{dc7ae26520315f,
title = {From Pixels to Pairs: A Comprehensive Benchmark of LLM-Driven Key-Value Extraction in Noisy Document Settings — arXiv:2609.17538},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.17538},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7ae26520315f; sha256 3bb32bb0d21239162125a8f984711b12b6b578b768874c92744235bf13a0e87c},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.17538. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language models (LLMs) have demonstrated strong capabilities in document key-value pair (KVP) extraction, yet controlled evaluations of their robustness to optical character recognition (OCR) output remain limited. This leaves an important gap in understanding their reliability in real-world OCR-to-LLM pipelines. Unlike end-to-end Vision-Language Models (VLMs), which jointly perform visual perception and semantic extraction, modular pipelines allow these stages and their errors to be isolated and audited. We introduce a controlled benchmark that distinguishes downstream LLM extraction behavior from upstream OCR degradation. It evaluates 136 experimental configurations and 17,688 document-level inferences generated with deterministic decoding across five instruction-tuned open-weight LLMs (2B-8B parameters), three datasets, and four text-quality conditions. The evaluation combines a full zero-shot comparison, targeted one- to three-shot experiments, and a sensitivity analysis of 40 configurations across 20 frozen demonstration sets. By separating Key Recall (annotated-field recovery) from Exact Match and Value F1 (exact and partial value recovery, respectively), we test whether OCR degradation affects field identification and value reproduction differently across models. Our findings challenge three practical assumptions: (1) clean-text performance reliably predicts real-world robustness, (2) model rankings remain consistent across annotation-derived Gold and OCR-derived text, and (3) additional few-shot demonstrations monotonically improve extraction accuracy. The observed model-ranking reversals and unstable few-shot gains expose important reliability risks under noisy document conditions. We release the benchmarking framework, dataset splits, and evaluation scripts to support reproducible research. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-17 in cs.CL). Categorie arXiv: Computation and Language (cs.CL) ; Computer Vision and Pattern Recognition (cs.CV). Pagina del lavoro: https://arxiv.org/abs/2609.17538 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-15T13:48:24.249Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-17?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).