Metraipolis › archive › dc_e20f57be93be

DecoyCount: Evaluating Visual-Reasoning Misalignment in Counting — arXiv:2610.34855

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.34855
License (as archived)
CC0-1.0
Archived
2026-10-27T07:58:18.470Z
SHA-256 of the archived text
1b868af0274efc1fb1657aa3a5ed95bcc7e51a64dfaf55a11d214ffcaad9dc44
Stable URL of this page
https://metraipolis.com/documento/dc_e20f57be93be

How to cite

DecoyCount: Evaluating Visual-Reasoning Misalignment in Counting — arXiv:2610.34855 — arXiv (en), https://arxiv.org/abs/2610.34855 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e20f57be93be (Metraipolis archive dc_e20f57be93be, sha256 1b868af0274efc1fb1657aa3a5ed95bcc7e51a64dfaf55a11d214ffcaad9dc44)
BibTeX
@misc{dce20f57be93be,
  title = {DecoyCount: Evaluating Visual-Reasoning Misalignment in Counting — arXiv:2610.34855},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.34855},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e20f57be93be; sha256 1b868af0274efc1fb1657aa3a5ed95bcc7e51a64dfaf55a11d214ffcaad9dc44},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.34855. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Visual counting benchmarks report whether vision-language models produce the correct number, but not whether errors arise from perception or reasoning. We introduce DecoyCount, a benchmark built from 1,660 real-world images with designed decoys across seven mechanisms, along with paired prompts: a neutral question, a matched alignment cue, and a novel multi-hop, multi-image question that situates the same counting problem in a realistic scenario. Using this design, we find that counting failures are induced partly by visual misalignment: models can execute the intended counting process when cue-guided but fail to select it unaided, and we term this difference in accuracy the reasoning-alignment gap. Despite being highly capable on general reasoning, frontier models remain below 50\% accuracy on our benchmark, while alignment gaps range from 11.8-22.1 percentage points; strikingly, we observe that more capable models, greater thinking budgets, and agentic processes often induce larger gaps. Because humans may bring context that models lack, we test whether textual underspecification contributes to misalignment using a scenario-based multi-hop condition without explicit hinting, but this added context does not improve aggregate performance. Our analysis concludes that stronger general reasoning does not by itself generalize to aligned visual reasoning, leaving misalignment with human visual expectations as a substantial component of the remaining deficit.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2610.34855 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-27T07:55:30.719Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).