Metraipolis › archive › dc_aa1f8ece8944

UVU: Improving Multimodal Understanding via Vision-Language Unified Autoregressive Paradigm — arXiv:2609.27915

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.27915
License (as archived)
CC0-1.0
Archived
2026-10-08T12:35:08.387Z
SHA-256 of the archived text
a693bf2a1940274fbe5990573fab299f63cd7742d61bcf7a65e1dde5e586b9e3
Stable URL of this page
https://metraipolis.com/documento/dc_aa1f8ece8944

How to cite

UVU: Improving Multimodal Understanding via Vision-Language Unified Autoregressive Paradigm — arXiv:2609.27915 — arXiv (en), https://arxiv.org/abs/2609.27915 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_aa1f8ece8944 (Metraipolis archive dc_aa1f8ece8944, sha256 a693bf2a1940274fbe5990573fab299f63cd7742d61bcf7a65e1dde5e586b9e3)
BibTeX
@misc{dcaa1f8ece8944,
  title = {UVU: Improving Multimodal Understanding via Vision-Language Unified Autoregressive Paradigm — arXiv:2609.27915},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.27915},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_aa1f8ece8944; sha256 a693bf2a1940274fbe5990573fab299f63cd7742d61bcf7a65e1dde5e586b9e3},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.27915. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Despite remarkable advancements in multimodal large language models (MLLMs), their fine-grained visual understanding is constrained by a primary reliance on sparse textual supervision. Existing efforts to introduce visual supervision typically do so during post-training, when visual representations have already been largely fixed, causing such signals to act mainly as auxiliary constraints rather than as a primary force for shaping perceptual features. In this paper, we aim to fundamentally reshape the model's perceptual backbone by incorporating vision supervision directly into the pre-training stage. We observe that pixel-level image patches and textual tokens naturally coexist in a shared, raw high-dimensional space characterized by an inherent input symmetry. Leveraging this insight, we propose UVU, a novel vision-language unified autoregressive framework that eschews vector quantization. It uniquely employs continuous visual encoding for lossless representation of visual inputs and proposes a large-scale iterative hierarchical clustering algorithm to construct a pixel-level visual codebook, thereby extending the vocabulary for unified supervision and enabling autoregressive generation of pixel-level image tokens alongside textual tokens. UVU effectively synergizes pixel-level visual perception with semantic-level visual understanding, internalizing visual reconstruction capabilities and unlocking the facilitative role of visual supervision in enhancing understanding in the pre-training stage. Extensive experiments across multiple tasks demonstrate that MLLMs are capable of achieving superior multimodal understanding performance under the supervised learning paradigm of UVU.

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-24 in cs.CV).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2609.27915 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T12:12:35.750Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-09-24?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).