Metraipolis › archive › dc_6453dcbe8bf2
Visual Grounding First, Multimodal In-context Learning Follows — arXiv:2610.30592
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.30592
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-23T15:15:00.608Z
- SHA-256 of the archived text
41daedef340ffa5b1e7f42e7ce7806508700680a421e2ba5b678c948efacdea0- Stable URL of this page
https://metraipolis.com/documento/dc_6453dcbe8bf2
How to cite
Visual Grounding First, Multimodal In-context Learning Follows — arXiv:2610.30592 — arXiv (en), https://arxiv.org/abs/2610.30592 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_6453dcbe8bf2 (Metraipolis archive dc_6453dcbe8bf2, sha256 41daedef340ffa5b1e7f42e7ce7806508700680a421e2ba5b678c948efacdea0)
BibTeX
@misc{dc6453dcbe8bf2,
title = {Visual Grounding First, Multimodal In-context Learning Follows — arXiv:2610.30592},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.30592},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_6453dcbe8bf2; sha256 41daedef340ffa5b1e7f42e7ce7806508700680a421e2ba5b678c948efacdea0},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.30592. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
LLMs have shown strong in-context learning (ICL) capability, but extending it to Multimodal LLMs (MLLMs) remains challenging. Prior multimodal ICL methods often rely on ICL-specific datasets for additional training or task vector extraction, which can improve performance on the same ICL benchmark used for adaptation. However, they do not generalize well to other benchmarks and often induce forgetting in previously well-solved tasks. For this reason, we identify 'visual grounding' as the key bottleneck in multimodal ICL; MLLMs often fail to attend to task-relevant visual evidence in demonstrations, instead over-relying on textual cues and producing hallucinated outputs. To address this, we propose ProCoRe to enhance visual grounding through contrastive reinforcement learning on multimodal contrastive data for improving multimodal ICL. ProCoRe generates captions for contrastive image pairs, decomposes them into propositions, and optimizes contrastive rewards that encourage alignment with corresponding images while discouraging mismatched ones. Despite never seeing ICL-formatted training examples, ProCoRe improves average multimodal ICL classification accuracy by 5.6% and captioning ROUGE-L by 13.8% over the off-the-shelf Qwen-VL-3-8B model, outperforming ICL-data-dependent state of the arts across benchmarks. Finally, we introduce PICL, a personalized multimodal ICL captioning benchmark that tests whether MLLMs genuinely understand visual demonstrations rather than relying on pre-trained semantic priors. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.30592 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-23T15:09:08.534Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).