Metraipolis › archive › dc_5ab180ffd386

DistilVDR: A Compact End-to-End Visual Document Retriever via Dual-Student Distillation — arXiv:2608.10636

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.10636
License (as archived)
CC0-1.0
Archived
2026-10-19T06:14:29.787Z
SHA-256 of the archived text
7a63212e8eb78d7371cb7210de55db2a5cb84931bd1dbf8ad783034539d4308d
Stable URL of this page
https://metraipolis.com/documento/dc_5ab180ffd386

How to cite

DistilVDR: A Compact End-to-End Visual Document Retriever via Dual-Student Distillation — arXiv:2608.10636 — arXiv (en), https://arxiv.org/abs/2608.10636 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_5ab180ffd386 (Metraipolis archive dc_5ab180ffd386, sha256 7a63212e8eb78d7371cb7210de55db2a5cb84931bd1dbf8ad783034539d4308d)
BibTeX
@misc{dc5ab180ffd386,
  title = {DistilVDR: A Compact End-to-End Visual Document Retriever via Dual-Student Distillation — arXiv:2608.10636},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.10636},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_5ab180ffd386; sha256 7a63212e8eb78d7371cb7210de55db2a5cb84931bd1dbf8ad783034539d4308d},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.10636. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Visual document retrieval (VDR) is dominated by multi-billion-parameter models that are slow to index at full corpus scale and expensive to serve. Prior compression routes either train a smaller multi-vector encoder from scratch or distil only the query side; neither yields a compact single-vector retriever end-to-end. We present DistilVDR, a 524M end-to-end VDR system distilled bilaterally from a single 8B vision-language teacher under a pointwise cosine alignment loss. All supervision comes from the frozen teacher's embedding space, which was itself trained with relevance supervision, so the student objective needs no relevance labels, negative sampling, or contrastive term. We match VDR's text-query and image-document input asymmetry with an asymmetric encoder-only student that concentrates visual capacity on the document side and keeps the query side at 70M parameters. We release two variants that share the same encoders and training and differ only in the document encoder's visual-tile budget: DistilVDR-HiRes attains 61.74 average NDCG@5 on ViDoRe v1+v2+v3 (86.9% of the 8B teacher) and leads every reproduced sub-1B baseline on the high-resolution-sensitive v3 benchmark, while DistilVDR-Fast attains 59.98 at a 3 times smaller visual-token budget. Both variants store one million documents in a 15.6 times smaller index than the strongest sub-1B multi-vector baseline and index the corpus an order of magnitude faster. The code is available at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-12 in cs.CL).
Categorie arXiv: Information Retrieval (cs.IR) ; Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2608.10636 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T06:08:50.362Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-12?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).