Metraipolis › archive › dc_7e42949f5837

Beyond Visual Similarity: Entity-Aligned Retrieval for Knowledge-Based Visual Question Answering — arXiv:2608.21450

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.21450
License (as archived)
CC0-1.0
Archived
2026-10-18T08:56:34.020Z
SHA-256 of the archived text
83fb5c631565115fe11977007dbedb4511c7bbaf385ceeb45dc261b5538831e4
Stable URL of this page
https://metraipolis.com/documento/dc_7e42949f5837

How to cite

Beyond Visual Similarity: Entity-Aligned Retrieval for Knowledge-Based Visual Question Answering — arXiv:2608.21450 — arXiv (en), https://arxiv.org/abs/2608.21450 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7e42949f5837 (Metraipolis archive dc_7e42949f5837, sha256 83fb5c631565115fe11977007dbedb4511c7bbaf385ceeb45dc261b5538831e4)
BibTeX
@misc{dc7e42949f5837,
  title = {Beyond Visual Similarity: Entity-Aligned Retrieval for Knowledge-Based Visual Question Answering — arXiv:2608.21450},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.21450},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7e42949f5837; sha256 83fb5c631565115fe11977007dbedb4511c7bbaf385ceeb45dc261b5538831e4},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.21450. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Knowledge-Based Visual Question Answering (KB-VQA) relies on retrieving external information to answer queries involving long-tail entities. However, existing retrieval pipelines predominantly employ CLIP-style dual encoders, which prioritize surface-level visual similarity over entity-level semantic alignment. This paradigm often fails when semantically identical concepts exhibit large visual variations or when distinct entities appear visually similar. To address this, we propose KBMR, the first MLLM-based embedding retriever tailored for KB-VQA. Leveraging the robust autoregressive capabilities of MLLMs, KBMR maps images into a semantic space that better preserves concept identity. To tackle the challenge of noisy supervision in Wikipedia-scale retrieval, we introduce an MLLM-based semantic discriminator that generates continuous entity-consistency weights. These weights guide a novel continuous semantic distillation objective, enabling effective hard negative sampling and soft supervision beyond rigid binary labels. Extensive experiments demonstrate that KBMR significantly outperforms CLIP baselines, yielding up to a 14.7% improvement in retrieval Recall@1 and a 9.4% gain in end-to-end VQA accuracy. Code is available at this https URL .

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-08-25 in cs.CV).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2608.21450 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T08:48:09.086Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-08-25?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).