Metraipolis › archive › dc_c819bf7c2b57
MoETok: Sparse Expert Routing for 1D Visual Tokenizers — arXiv:2610.41992
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.41992
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-02T05:04:19.160Z
- SHA-256 of the archived text
4e2471e23d14603132ee053eb035fc1b01c94ed185933241e9a81a5845351164- Stable URL of this page
https://metraipolis.com/documento/dc_c819bf7c2b57
How to cite
MoETok: Sparse Expert Routing for 1D Visual Tokenizers — arXiv:2610.41992 — arXiv (en), https://arxiv.org/abs/2610.41992 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_c819bf7c2b57 (Metraipolis archive dc_c819bf7c2b57, sha256 4e2471e23d14603132ee053eb035fc1b01c94ed185933241e9a81a5845351164)
BibTeX
@misc{dcc819bf7c2b57,
title = {MoETok: Sparse Expert Routing for 1D Visual Tokenizers — arXiv:2610.41992},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.41992},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_c819bf7c2b57; sha256 4e2471e23d14603132ee053eb035fc1b01c94ed185933241e9a81a5845351164},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.41992. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Modern visual tokenizers aim to preserve fine-grained visual details while learning high-level semantic representations for downstream generation and understanding. Query-based 1D visual tokenizers encode images into flexible latent sequences, yet how individual learnable queries gather and encode image information remains poorly understood. We reveal that these queries spontaneously develop complementary roles under semantic supervision: a small set of Content queries captures high-level semantic information, while a larger set of Detail queries preserves fine-grained visual details for reconstruction. To adapt token processing to these heterogeneous roles, we introduce MoETok, a 1D visual tokenizer that routes tokens through sparsely activated experts. Jointly optimized with reconstruction and semantic objectives, the router learns role-specialized processing paths adapted to each token's representation. Across model scales and token budgets, MoETok consistently improves reconstruction, semantic representation, and autoregressive generation over dense tokenizers under matched supervision and active computation. On ImageNet, it combines strong reconstruction and semantic representation with state-of-the-art class-conditional generation within the 1D tokenizer family. We further validate MoETok at scale in a unified autoregressive model jointly trained for understanding and generation with a next-token cross-entropy objective. The resulting model combines competitive visual understanding with state-of-the-art generation among unified discrete models, matching or surpassing diffusion-based models such as BAGEL and Qwen-Image on benchmarks including DPG-Bench (87.61), GenEval2 (40.04 GM), and WISE (0.49). Tema degli esploratori: statistica e scienza dei dati (elenco arXiv cs.CV, nuovi lavori del giorno). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV). Pagina del lavoro: https://arxiv.org/abs/2610.41992 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-02T04:49:13.280Z dalla pagina https://arxiv.org/list/cs.CV/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).