Metraipolis › archive › dc_49c13d37c71f

YANAMI: A Large-Scale Multimodal Dataset and Benchmark for Multiturn Stateful Interaction — arXiv:2611.05280

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.05280
License (as archived)
CC0-1.0
Archived
2026-11-04T12:57:28.846Z
SHA-256 of the archived text
1305d6a937f7176f0d5b34033bd2dde389df0b3e480b373ff39f4c84cae3d194
Stable URL of this page
https://metraipolis.com/documento/dc_49c13d37c71f

How to cite

YANAMI: A Large-Scale Multimodal Dataset and Benchmark for Multiturn Stateful Interaction — arXiv:2611.05280 — arXiv (en), https://arxiv.org/abs/2611.05280 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_49c13d37c71f (Metraipolis archive dc_49c13d37c71f, sha256 1305d6a937f7176f0d5b34033bd2dde389df0b3e480b373ff39f4c84cae3d194)
BibTeX
@misc{dc49c13d37c71f,
  title = {YANAMI: A Large-Scale Multimodal Dataset and Benchmark for Multiturn Stateful Interaction — arXiv:2611.05280},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.05280},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_49c13d37c71f; sha256 1305d6a937f7176f0d5b34033bd2dde389df0b3e480b373ff39f4c84cae3d194},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.05280. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Unified Multimodal Models (UMMs) integrate visual understanding, generation, and editing within a single unified framework. However, existing unified multimodal datasets and benchmarks remain dominated by single-turn tasks. Even in recent multi-turn datasets, many turns can be solved independently without tracking information across turns. Consequently, they provide limited supervision and evaluation for multimodal stateful interaction, which requires models to understand, maintain, update, and faithfully realize evolving dialogue states across turns. To address this gap, we introduce YANAMI, the first large-scale multimodal dialogue dataset and benchmark for stateful interaction, comprising YANAMI-400K and YANAMI-Bench. YANAMI-400K contains 400K multi-turn multimodal dialogues, with more than 2M images and 2.6M dialogue turns. The dialogues span four representative scenarios: Multi-Image Fusion, Image Editing, World Modeling, and Visual Reasoning, and require models to continuously track, preserve, and revise evolving multimodal dialogue states across turns. YANAMI-Bench comprises 400 multi-turn dialogues for systematically evaluating multimodal stateful interaction. It jointly evaluates Dialogue State Understanding and Visual Realization, characterizing the gap between models' ability to understand evolving dialogue states and their ability to faithfully realize them in visual outputs. Experiments show that fine-tuning BAGEL on YANAMI-400K improves image editing, generation, and multimodal stateful interaction by 10.3%, 2.0%, and 13.23%, respectively, while preserving its general visual understanding capabilities. Evaluations on YANAMI-Bench further reveal that existing UMMs still struggle to maintain, update, and visually realize evolving multimodal dialogue states.

Tema degli esploratori: statistica e scienza dei dati (elenco arXiv cs.CV, nuovi lavori del giorno).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2611.05280 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-04T12:33:47.231Z dalla pagina https://arxiv.org/list/cs.CV/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).