Metraipolis › archive › dc_70210220df02
Revisiting Visual Representation Enhancement of VLMs via Kernel Canonical Correlation Analysis — arXiv:2610.02718
- Source
- arXiv
- Original
- https://arxiv.org/abs/2610.02718
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-05T22:50:13.323Z
- SHA-256 of the archived text
c471806977b1d1783d945581ea9462842eaf289c9bbafdbd86afbf0b36bd3f7c- Stable URL of this page
https://metraipolis.com/documento/dc_70210220df02
How to cite
Revisiting Visual Representation Enhancement of VLMs via Kernel Canonical Correlation Analysis — arXiv:2610.02718 — arXiv, https://arxiv.org/abs/2610.02718 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_70210220df02 (Metraipolis archive dc_70210220df02, sha256 c471806977b1d1783d945581ea9462842eaf289c9bbafdbd86afbf0b36bd3f7c)
BibTeX
@misc{dc70210220df02,
title = {Revisiting Visual Representation Enhancement of VLMs via Kernel Canonical Correlation Analysis — arXiv:2610.02718},
howpublished = {arXiv},
url = {https://arxiv.org/abs/2610.02718},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_70210220df02; sha256 c471806977b1d1783d945581ea9462842eaf289c9bbafdbd86afbf0b36bd3f7c},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.02718. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Vision-language models such as CLIP exhibit strong semantic generalization, but remain limited in fine-grained visual perception. A recent work named KUEA presents a natural remedy by finetuning the image encoder under the supervision of the vision-centric DINOv2 to align their kernel matrices element-wisely, while regularizing the embeddings to remain close to the pretrained visual encoder for preserving image-text semantics in CLIP. However, we show that diminishing the role of the alignment loss to DINOv2 does not necessarily degrade its fine-grained visual performance, suggesting that the kernel-matrix discrepancy may be insufficient for further visual representation enhancement, motivating us to revisit the alignment formulation. In this work, we present a novel perspective to characterize representation alignment on feature subspaces through Kernel Canonical Correlation Analysis (KCCA), which maximizes the projection correlations. In optimization, we derive an efficient end-to-end training scheme upon KKT conditions, avoiding the eigenvalue problem in KCCA. Further, we extend our method into a 3-view formulation, i.e., 3vKCCA, in which the projections from the pretrained text encoder are also incorporated under a unified optimization framework for joint alignment. With CLIP ViT-L/14 on ImageNet-1K, our 3vKCCA improves the MMVP-VLM accuracy from 17.8 to 25.9, substantially outperforming the existing methods, and meanwhile maintains zero-shot image--text retrieval performance. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2610.02718 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-05T22:45:55.790Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).