Metraipolis › archive › dc_f3fd2078feb2

Decoupled Mode Connectivity for Base-to-Novel Generalization in Vision-Language Models — arXiv:2611.09655

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.09655
License (as archived)
CC0-1.0
Archived
2026-11-09T22:45:24.946Z
SHA-256 of the archived text
b93739c780a94413a63f85ad2029f13fd70769e642c518effb7e9ea719fda42e
Stable URL of this page
https://metraipolis.com/documento/dc_f3fd2078feb2

How to cite

Decoupled Mode Connectivity for Base-to-Novel Generalization in Vision-Language Models — arXiv:2611.09655 — arXiv (en), https://arxiv.org/abs/2611.09655 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f3fd2078feb2 (Metraipolis archive dc_f3fd2078feb2, sha256 b93739c780a94413a63f85ad2029f13fd70769e642c518effb7e9ea719fda42e)
BibTeX
@misc{dcf3fd2078feb2,
  title = {Decoupled Mode Connectivity for Base-to-Novel Generalization in Vision-Language Models — arXiv:2611.09655},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.09655},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f3fd2078feb2; sha256 b93739c780a94413a63f85ad2029f13fd70769e642c518effb7e9ea719fda42e},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.09655. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Prompt learning adapts vision-language models such as CLIP by optimizing a short learnable prompt. The cross-entropy objective pulls this prompt toward base-class specialization, whereas generalization to unseen classes benefits from staying close to CLIP's zero-shot feature space. Because both objectives act on the same prompt, their gradients conflict at convergence, and in our experiments no single-prompt loss design clearly improves on the cosine baseline. We propose Decoupled Mode Connectivity (DMC), which gives each objective its own prompt. Starting from a base-trained prompt, DMC trains a specialization prompt and a generalization prompt, and merges their text features with a fixed coefficient at test time, at the inference cost of a single prompt. A linear mode connectivity (LMC) corridor in text-feature space keeps the classification loss low for every classifier interpolated between the two endpoints, and a Visual Anchor KL term preserves CLIP's zero-shot class-similarity distribution of each image during specialization. We further argue that such regularizers should satisfy class-permutation invariance (CPI), and show via Fano's inequality, in an idealized model, that violating CPI lower-bounds the drop in novel accuracy by a term proportional to the mutual information between the prompt and the base-class labels. Across 11 datasets, DMC improves the average harmonic mean of base and novel accuracy over MergeTune for both CoOp and KgCoOp, and its interpolation curves pass beyond the single-prompt operating point, shifting the base-novel frontier rather than trading along it. Code is available at this https URL .

Tema degli esploratori: statistica e scienza dei dati (elenco arXiv cs.CV, nuovi lavori del giorno).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2611.09655 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-09T22:37:47.231Z dalla pagina https://arxiv.org/list/cs.CV/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).