Metraipolis › archive › dc_e79400ec0300

Multimodal Flow: Unified Flow Modeling of Language and Vision in Embedding Spaces — arXiv:2609.40362

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.40362
License (as archived)
CC0-1.0
Archived
2026-10-07T14:46:43.358Z
SHA-256 of the archived text
d6cd20c988e65b4cc342ae587212b2d1e424da3e794827061f78fb801e53c6c5
Stable URL of this page
https://metraipolis.com/documento/dc_e79400ec0300

How to cite

Multimodal Flow: Unified Flow Modeling of Language and Vision in Embedding Spaces — arXiv:2609.40362 — arXiv (en), https://arxiv.org/abs/2609.40362 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e79400ec0300 (Metraipolis archive dc_e79400ec0300, sha256 d6cd20c988e65b4cc342ae587212b2d1e424da3e794827061f78fb801e53c6c5)
BibTeX
@misc{dce79400ec0300,
  title = {Multimodal Flow: Unified Flow Modeling of Language and Vision in Embedding Spaces — arXiv:2609.40362},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.40362},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e79400ec0300; sha256 d6cd20c988e65b4cc342ae587212b2d1e424da3e794827061f78fb801e53c6c5},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.40362. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

We present Multimodal Flow, a fully continuous generative model of language and vision. Most unified multimodal models either model both language and quantized images as discrete tokens or combine discrete language prediction with continuous image generation. The former introduces a visual quantization bottleneck. The latter requires modality-dependent objectives and sampling procedures. Fully continuous modeling avoids these trade-offs and enables a shared generative process, but remains underexplored for multimodal pretraining. Multimodal Flow introduces a unified continuous architecture that integrates multimodal continuous representations with a shared chunk-causal flow backbone. It organizes text blocks and images as ordered continuous hyperchunks, preserving textual token order and visual spatial structure. The backbone learns a single vector field over these hyperchunks through Flow Matching. Joint attention enables cross-modal interaction, while modality-specific feed-forward networks process each modality. The model predicts multiple target chunks in parallel during training and generates hyperchunks sequentially at inference. We instantiate MF-1 and pretrain it on multimodal data. Across 0.6B, 1.2B, and 1.6B scales, continued pretraining consistently improves multimodal modeling. With only 150B pretraining tokens, MF-1 achieves an average score of 82.8 across GenEval and DPG-Bench and 75.3 across VQAv2, MMBench, and POPE, remaining competitive with unified models trained on substantially more data. Under matched data, optimization, and parameter budgets, Multimodal Flow further outperforms representative hybrid and discrete models. These results establish continuous chunk-based embedding flow modeling as a new fully continuous paradigm for unified multimodal modeling. The related code and model are publicly released at this https URL .

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-10-01 in cs.CV).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2609.40362 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T14:19:00.872Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-10-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).