Metraipolis › archive › dc_8aab8b23b167

Can Visual Generation Expand What Unified Multimodal Models Know? — arXiv:2610.21459

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.21459
License (as archived)
CC0-1.0
Archived
2026-10-16T09:24:05.871Z
SHA-256 of the archived text
24521ba057d72858d687b4118f5d49f989744c697d3cd8177beeb1708c121bde
Stable URL of this page
https://metraipolis.com/documento/dc_8aab8b23b167

How to cite

Can Visual Generation Expand What Unified Multimodal Models Know? — arXiv:2610.21459 — arXiv (en), https://arxiv.org/abs/2610.21459 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_8aab8b23b167 (Metraipolis archive dc_8aab8b23b167, sha256 24521ba057d72858d687b4118f5d49f989744c697d3cd8177beeb1708c121bde)
BibTeX
@misc{dc8aab8b23b167,
  title = {Can Visual Generation Expand What Unified Multimodal Models Know? — arXiv:2610.21459},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.21459},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_8aab8b23b167; sha256 24521ba057d72858d687b4118f5d49f989744c697d3cd8177beeb1708c121bde},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.21459. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Unified multimodal models (UMMs) receive image-generation supervision that exposes them to visual information not fully documented in text. Can learning to generate images expand what pretrained models know? We introduce Knowledge Transfer Probing (KTP), a controlled protocol that supplies new knowledge through image-generation training and tests its accessibility through text-only question answering. Across seven knowledge categories and three UMMs, we observe direct but uneven transfer from visual supervision to language. Motivated by these observations, we successively introduce Textual Alignment and Interleaved Image--Text Alignment, which connect generation and answering through shared knowledge and interleaved responses, respectively. Guidance on selected entities can improve knowledge access on other entities without QA supervision, with additional gains from interleaved alignment. Further analysis shows that knowledge correctly reproduced in images is often more accessible through text, although substantial gaps remain between the two forms of access. These findings support visual generation as a source of knowledge beyond textual supervision, and motivate joint investigation of training data design and model architecture to make this knowledge more broadly accessible.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.21459 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T08:44:17.954Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).