Metraipolis › archive › dc_64f2bfb6cfa4
Hyper-LLaVA: Hyperbolic Uncertainty-aware Modality-Balanced Routing for Multimodal Continual Instruction Tuning — arXiv:2609.13742
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.13742
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T01:52:43.011Z
- SHA-256 of the archived text
9bc015c086b923b71b46d6777957e05a9fbf7b0f8f62dd4b0c52d633966de72a- Stable URL of this page
https://metraipolis.com/documento/dc_64f2bfb6cfa4
How to cite
Hyper-LLaVA: Hyperbolic Uncertainty-aware Modality-Balanced Routing for Multimodal Continual Instruction Tuning — arXiv:2609.13742 — arXiv (en), https://arxiv.org/abs/2609.13742 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_64f2bfb6cfa4 (Metraipolis archive dc_64f2bfb6cfa4, sha256 9bc015c086b923b71b46d6777957e05a9fbf7b0f8f62dd4b0c52d633966de72a)
BibTeX
@misc{dc64f2bfb6cfa4,
title = {Hyper-LLaVA: Hyperbolic Uncertainty-aware Modality-Balanced Routing for Multimodal Continual Instruction Tuning — arXiv:2609.13742},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.13742},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_64f2bfb6cfa4; sha256 9bc015c086b923b71b46d6777957e05a9fbf7b0f8f62dd4b0c52d633966de72a},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.13742. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Multimodal Continual Instruction Tuning (MCIT) aims to exploit the incrementally accumulated knowledge to process multimodal inputs of diverse tasks, where parameter routing plays an important role. State-of-the-art methods rely on sample-to-task center similarity and cross-modal fusion with equal weight during routing. However, such solutions face two fundamental flaws: (1) Within each modality, the sample-to-task center distance is sub-optimal for routing since the abundant intra-task diversity information is underleveraged. (2) Different modalities exhibit varying reliability across tasks, where the modality with inter-task ambiguity can easily misguide the routing result. To address these problems, we propose Hyperbolic Uncertainty-aware Modality-Balanced Routing (Hyper-LLaVA) to improve parameter routing capacity based on cross-modality task feature uncertainty modeling. Specifically, to improve intra-modality task matching, Hyper-LLaVA accesses the sample-to-task distribution similarity in the Hyperbolic space. Besides, to alleviate the degradation brought by unreliable modalities, Hyper-LLaVA quantifies the task matching ambiguity within each modality to achieve adaptive balancing between task matching across modalities. Based on the complementary intra- and inter-modality task matching enhancement, our Hyper-LLaVA outperforms state-of-the-art approaches by large margins. Our source code is available at this https URL Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-15 in cs.CV). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV). Pagina del lavoro: https://arxiv.org/abs/2609.13742 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T01:25:23.521Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-09-15?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).