Metraipolis › archive › dc_b0ba0cbb63be

Do Multimodal LLMs See Before They Read? Diagnosing Contextual Sycophancy — arXiv:2609.00067

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.00067
License (as archived)
CC0-1.0
Archived
2026-10-17T06:32:41.992Z
SHA-256 of the archived text
e74f199ee5b7b75eb7caa9cec1e73a7b895e1253347c3adf6fc678d0f8229664
Stable URL of this page
https://metraipolis.com/documento/dc_b0ba0cbb63be

How to cite

Do Multimodal LLMs See Before They Read? Diagnosing Contextual Sycophancy — arXiv:2609.00067 — arXiv (en), https://arxiv.org/abs/2609.00067 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b0ba0cbb63be (Metraipolis archive dc_b0ba0cbb63be, sha256 e74f199ee5b7b75eb7caa9cec1e73a7b895e1253347c3adf6fc678d0f8229664)
BibTeX
@misc{dcb0ba0cbb63be,
  title = {Do Multimodal LLMs See Before They Read? Diagnosing Contextual Sycophancy — arXiv:2609.00067},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.00067},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b0ba0cbb63be; sha256 e74f199ee5b7b75eb7caa9cec1e73a7b895e1253347c3adf6fc678d0f8229664},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.00067. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

External text can override conflicting image evidence in multimodal large language models, a failure we call multimodal contextual sycophancy. We introduce a 998-case diagnostic that independently varies visual evidence, commonsense priors, and external text, and probe when this failure arises by moving the information boundary around a context-blind visual witness. On abnormal images paired with Gemini-generated false text, GPT-5.1 scores 7.9% under joint conditioning, 49.7% when the context-blind witness report is scored directly, 63.7% under a matched two-call witness-arbiter pipeline that exposes the witness to the text, and 84.2% under System-2 Visual Arbitration (S2VA), which withholds the text from the witness. Across six models, S2VA improves over the direct witness report by 19.7 to 44.1 points, with all paired 95% confidence intervals excluding zero. The best information boundary is not uniform: textual context scaffolds some models, and a GPT-4o-regenerated subset changes the relative ordering of joint conditioning, Witness-Only, and S2VA. Contextual sycophancy is therefore sensitive to when text is introduced, as well as to the model and context source.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-02 in cs.AI).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.00067 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T06:15:12.731Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).