Metraipolis › archive › dc_4ce3b47484c8

DEEPO: Dual-Entropy Enhanced Policy Optimization for Hallucination in MLLMs — arXiv:2609.28570

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.28570
License (as archived)
CC0-1.0
Archived
2026-10-08T04:25:38.218Z
SHA-256 of the archived text
ed72605cae22d4c21fc026623139db5e8b2a739ee7c3c41b8ec02d85251dbba8
Stable URL of this page
https://metraipolis.com/documento/dc_4ce3b47484c8

How to cite

DEEPO: Dual-Entropy Enhanced Policy Optimization for Hallucination in MLLMs — arXiv:2609.28570 — arXiv (en), https://arxiv.org/abs/2609.28570 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4ce3b47484c8 (Metraipolis archive dc_4ce3b47484c8, sha256 ed72605cae22d4c21fc026623139db5e8b2a739ee7c3c41b8ec02d85251dbba8)
BibTeX
@misc{dc4ce3b47484c8,
  title = {DEEPO: Dual-Entropy Enhanced Policy Optimization for Hallucination in MLLMs — arXiv:2609.28570},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.28570},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4ce3b47484c8; sha256 ed72605cae22d4c21fc026623139db5e8b2a739ee7c3c41b8ec02d85251dbba8},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.28570. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Reinforcement learning (RL) is widely used to sharpen reasoning in multimodal large language models (MLLMs), yet its effect on hallucination is uneven. We trace this to two weak points in the \emph{correction chain} from reward to parameter update. At the rollout level, hard queries---those with high semantic entropy---frequently produce unanimously wrong sample groups, collapsing the group-relative advantage to zero exactly where hallucination risk is highest. At the optimization level, confident-but-wrong tokens are gradient-invisible: a categorical policy's expected score-gradient norm vanishes as its distribution sharpens, so the predictions that most need correction receive the weakest updates. We propose Dual-Entropy Enhanced Policy Optimization (DEEPO), a dual-stage enhancement combining signal variance regularization with gradient preconditioning: semantic-entropy-triggered expert prefixes inject grounded continuations on high-uncertainty queries, providing direct supervision and restoring advantage variance, while advantage-sign-aware Renyi preconditioning counteracts logit-level saturation so correction reaches confident errors in the operational confidence regime. Both branches improve over GRPO individually; their interaction is statistically significant on VideoMMMU---the most complex long-horizon task in our evaluation suite (+4.0$, 95\% CI [1.1, 6.9])---and additive elsewhere. DEEPO reduces hallucination while preserving accuracy and training stability.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-25 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.28570 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T03:50:57.656Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-25?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).