Metraipolis › archive › dc_cea99f5a8beb

Beyond Correctness: Benchmarking and Aligning Response Behaviors in Hybrid-Thinking MLLMs — arXiv:2608.12781

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.12781
License (as archived)
CC0-1.0
Archived
2026-10-19T04:55:36.800Z
SHA-256 of the archived text
81a15f969739a76517d9bce4ee05a6a2e8754b7a164cf4b6af73c8f8f18a4c3f
Stable URL of this page
https://metraipolis.com/documento/dc_cea99f5a8beb

How to cite

Beyond Correctness: Benchmarking and Aligning Response Behaviors in Hybrid-Thinking MLLMs — arXiv:2608.12781 — arXiv (en), https://arxiv.org/abs/2608.12781 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_cea99f5a8beb (Metraipolis archive dc_cea99f5a8beb, sha256 81a15f969739a76517d9bce4ee05a6a2e8754b7a164cf4b6af73c8f8f18a4c3f)
BibTeX
@misc{dccea99f5a8beb,
  title = {Beyond Correctness: Benchmarking and Aligning Response Behaviors in Hybrid-Thinking MLLMs — arXiv:2608.12781},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.12781},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_cea99f5a8beb; sha256 81a15f969739a76517d9bce4ee05a6a2e8754b7a164cf4b6af73c8f8f18a4c3f},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.12781. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Hybrid-thinking multimodal large language models (MLLMs) allow a single model to alternate between deliberative thinking and latency-efficient non-thinking inference. Although these modes differ in reasoning budget, their delivered responses should satisfy the same user-facing standard. Correctness alone may not characterize this response quality; we therefore evaluate task accuracy and response-pattern failures as complementary outcomes. We study this gap through \textbf{response-pattern alignment}: whether thinking and non-thinking interfaces preserve acceptable final-response behavior. We introduce \textbf{PatternEval}, a failure-enriched diagnostic benchmark comprising 2,415 multimodal prompts spanning visual perception and grounding, structured image understanding, and multimodal knowledge reasoning. PatternEval tests four recurrent failures: chain-of-thought leakage, response repetition, logical contradiction, and performative reasoning. Response-pattern failures are widespread across models from different providers, with non-thinking inference exhibiting substantially higher failure rates and thereby creating systematic misalignment between thinking and non-thinking interfaces. Motivated by this diagnosis, we develop \textbf{PatternRM}, a response-level reward model, and \textbf{PatternRL}, which introduces pattern-specific penalties during reinforcement learning. Experiments on Qwen3-VL-4B and Qwen3-VL-8B show that incorporating pattern-specific penalties into reinforcement learning can mitigate cross-mode misalignment while incurring a marginal task performance trade-off. Together, PatternEval and PatternRL provide an evaluation-and-training framework for aligning user-visible response patterns across hybrid-thinking interfaces.

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-08-14 in cs.CV).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2608.12781 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T04:46:30.063Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-08-14?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).