Metraipolis › archive › dc_9b4331b09b01

NeoMME: A Single-Tower Multimodal-Native Multilingual Foundation Encoder for Efficient Fine-Tuning and Inference — arXiv:2609.01657

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.01657
License (as archived)
CC0-1.0
Archived
2026-10-17T03:26:46.132Z
SHA-256 of the archived text
3594bace82b8d877b266afd817fbf756c728dfb9963f42c104869ee1dabec425
Stable URL of this page
https://metraipolis.com/documento/dc_9b4331b09b01

How to cite

NeoMME: A Single-Tower Multimodal-Native Multilingual Foundation Encoder for Efficient Fine-Tuning and Inference — arXiv:2609.01657 — arXiv (en), https://arxiv.org/abs/2609.01657 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_9b4331b09b01 (Metraipolis archive dc_9b4331b09b01, sha256 3594bace82b8d877b266afd817fbf756c728dfb9963f42c104869ee1dabec425)
BibTeX
@misc{dc9b4331b09b01,
  title = {NeoMME: A Single-Tower Multimodal-Native Multilingual Foundation Encoder for Efficient Fine-Tuning and Inference — arXiv:2609.01657},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.01657},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_9b4331b09b01; sha256 3594bace82b8d877b266afd817fbf756c728dfb9963f42c104869ee1dabec425},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.01657. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Multimodal models often build on architectures designed for generative vision-language modeling, typically combining separately pretrained vision encoders with causal language models. Visual document retrievers such as ColPali repurpose these models as encoders, carrying over the parameter and compute overhead of a VLM for a non-generative task. We introduce NeoMME, a family of 260M and 800M-parameter Multimodal and Multilingual bidirectional Encoders that process multilingual text and raw image patches in a single bidirectional Transformer encoder. Both models are pretrained from scratch with a masked discrete-diffusion text objective, conditioned on visible image patches for multimodal examples. Both support a 16,384-token context, enough to encode up to two standard 4K UHD images. To demonstrate its downstream capabilities, we fine-tune NeoMME with jointly trained dense and late-interaction heads. On the ViDoRe v3 benchmark, the resulting NeoMME-Retriever 260M outperforms all evaluated models strictly below 800M parameters with 0.523 nDCG@10, while NeoMME-Retriever 800M reaches 0.556. At a matched 2048x2048 image input size on an NVIDIA L40S, NeoMME-260M encodes pages with about 2x the throughput of ColModernVBERT. Hierarchical token pooling and asymmetric quantization compress late-interaction multimodal document embeddings by 255x while preserving over 95% of baseline nDCG@10. We contribute NeoMME to Hugging Face Transformers and release the pretrained backbone and retrieval-compatible checkpoints under Apache 2.0 at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-03 in cs.AI).
Categorie arXiv: Information Retrieval (cs.IR) ; Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2609.01657 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T03:13:12.466Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-03?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).