Metraipolis › archive › dc_54ae2ca8ee72
Can a Lightweight Multimodal Model Estimate LLM Reasoning Performance? A Study for Compute-Optimal Document Inference — arXiv:2608.18591
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.18591
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-18T13:44:32.909Z
- SHA-256 of the archived text
88c21b80cf284605b5a1c9dd8e82ed3de68d169712a95260647be70d89139f61- Stable URL of this page
https://metraipolis.com/documento/dc_54ae2ca8ee72
How to cite
Can a Lightweight Multimodal Model Estimate LLM Reasoning Performance? A Study for Compute-Optimal Document Inference — arXiv:2608.18591 — arXiv (en), https://arxiv.org/abs/2608.18591 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_54ae2ca8ee72 (Metraipolis archive dc_54ae2ca8ee72, sha256 88c21b80cf284605b5a1c9dd8e82ed3de68d169712a95260647be70d89139f61)
BibTeX
@misc{dc54ae2ca8ee72,
title = {Can a Lightweight Multimodal Model Estimate LLM Reasoning Performance? A Study for Compute-Optimal Document Inference — arXiv:2608.18591},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.18591},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_54ae2ca8ee72; sha256 88c21b80cf284605b5a1c9dd8e82ed3de68d169712a95260647be70d89139f61},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.18591. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Uniformly allocating inference reasoning budgets to LLMs is expensive and prone to over-thinking penalties; especially in document tasks where visual layouts drive complexity. To address this, we introduce BudgetDoc, the first multimodal benchmark providing explicit supervision for model-budget-performance trade-offs across three document tasks. Using BudgetDoc, we train DRB (Document-Reasoning Balancer), an approx. 1B-parameter pre-flight estimator (SigLIP-2 + Qwen3-0.6B) that predicts ordinal model performance across budget levels, achieving a 0.753 weighted F1. When dynamically allocating reasoning budgets across five frontier models and three datasets, DRB matches or improves F1 scores compared to always-maximum-budget baselines in 9 of 15 configurations while drastically reducing cost. Finally, preliminary evaluations demonstrate DRB's potential to generalize to cross-model selection. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-20 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2608.18591 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-18T13:41:17.869Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-20?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).