Metraipolis › archive › dc_a6574c415e07

ReWEIGH the Evidence: Calibrating Token-Level Ordinal Visual Evidence to Mitigate Hallucinations in Large Vision-Language Models — arXiv:2608.19075

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.19075
License (as archived)
CC0-1.0
Archived
2026-10-18T13:45:33.160Z
SHA-256 of the archived text
ce416b6ab56a5d9d09df8efc47c8673c200bcb60cb0f8cfae12e27653f416337
Stable URL of this page
https://metraipolis.com/documento/dc_a6574c415e07

How to cite

ReWEIGH the Evidence: Calibrating Token-Level Ordinal Visual Evidence to Mitigate Hallucinations in Large Vision-Language Models — arXiv:2608.19075 — arXiv (en), https://arxiv.org/abs/2608.19075 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a6574c415e07 (Metraipolis archive dc_a6574c415e07, sha256 ce416b6ab56a5d9d09df8efc47c8673c200bcb60cb0f8cfae12e27653f416337)
BibTeX
@misc{dca6574c415e07,
  title = {ReWEIGH the Evidence: Calibrating Token-Level Ordinal Visual Evidence to Mitigate Hallucinations in Large Vision-Language Models — arXiv:2608.19075},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.19075},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a6574c415e07; sha256 ce416b6ab56a5d9d09df8efc47c8673c200bcb60cb0f8cfae12e27653f416337},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.19075. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large vision-language models (LVLMs) often hallucinate, generating content that the input image does not support. Preventing such content during decoding calls for a candidate-specific measure of how strongly the image supports the token under consideration. The model's visual-token states offer a natural source of this evidence because projecting each state through the output head reveals which vocabulary items that position favors. These position-wise readouts cannot be pooled directly because their probability magnitudes are not comparable across visual positions. Vocabulary ranks provide a scale-invariant basis for pooling, but tokens still differ systematically in their typical rank-based evidence. We propose ReWEIGH, a training-free decoding intervention that aggregates these ranks across visual positions and compares each candidate with a token-specific reference estimated from unlabeled images. At inference, ReWEIGH caches the image evidence during prefill and applies a bounded penalty only to candidates that fall below their reference. On four 7B backbones, ReWEIGH reduces hallucinated object mentions by up to 21.3% while largely preserving or improving descriptive and general performance. With evidence cached, the average added latency is 1.33% per token, and the reductions extend across six architecture families to 32B parameters.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-20 in cs.AI).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2608.19075 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T13:41:17.997Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-20?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).