Metraipolis › archive › dc_47107f82dbbe
PEA-DPO: Perception-Enhanced Alignment Direct Preference Optimization for MLLMs Alignment — arXiv:2608.19598
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.19598
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-18T11:39:28.390Z
- SHA-256 of the archived text
204d2abba1218c6bbb44bd9de64e39db08649b2324184b8854604d1a6eeb496a- Stable URL of this page
https://metraipolis.com/documento/dc_47107f82dbbe
How to cite
PEA-DPO: Perception-Enhanced Alignment Direct Preference Optimization for MLLMs Alignment — arXiv:2608.19598 — arXiv (en), https://arxiv.org/abs/2608.19598 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_47107f82dbbe (Metraipolis archive dc_47107f82dbbe, sha256 204d2abba1218c6bbb44bd9de64e39db08649b2324184b8854604d1a6eeb496a)
BibTeX
@misc{dc47107f82dbbe,
title = {PEA-DPO: Perception-Enhanced Alignment Direct Preference Optimization for MLLMs Alignment — arXiv:2608.19598},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.19598},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_47107f82dbbe; sha256 204d2abba1218c6bbb44bd9de64e39db08649b2324184b8854604d1a6eeb496a},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.19598. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Direct Preference Optimization (DPO) has emerged as an effective approach for aligning large language models (LLMs) with human preferences. However, its adaptation to multimodal settings remains unexplored. Through representational analysis, we identify a key limitation in multimodal preference optimization, which we term visual insensitivity: models often fail to distinguish between images and those with critical visual context removed. Our theoretical analysis further uncovers two manifestations of this problem, namely Across-Image Insensitivity and Within-Image Insensitivity. To address these challenges, we propose Perception-Enhanced Alignment DPO (PEA-DPO), a framework for multimodal LLMs alignment, which explicitly leverages visual preference signals to overcome visual insensitivity. We further provide a theoretical analysis demonstrating that PEA-DPO provably mitigates both failure modes. Empirical results demonstrate that PEA-DPO enhances sensitivity to visual context while preserving the language modeling capacity of the base model. Evaluations across three hallucination benchmarks using MLLMs of varying scales show that PEA-DPO effectively mitigates visual insensitivity, achieves stronger multimodal alignment, and substantially reduces hallucinations. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-21 in cs.CL). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Multimedia (cs.MM). Pagina del lavoro: https://arxiv.org/abs/2608.19598 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-18T11:38:27.944Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-21?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).