Metraipolis › archive › dc_c585512db487
Which Negatives Matter? Ask Your Text Encoder: Adaptive Similarity Margins for Dense-Caption Retrieval — arXiv:2608.18521
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.18521
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-18T13:44:30.453Z
- SHA-256 of the archived text
ec81c99d7fb371502743e9008089ee7b3f273bee970bf2d9ddf138668fd9c19c- Stable URL of this page
https://metraipolis.com/documento/dc_c585512db487
How to cite
Which Negatives Matter? Ask Your Text Encoder: Adaptive Similarity Margins for Dense-Caption Retrieval — arXiv:2608.18521 — arXiv (en), https://arxiv.org/abs/2608.18521 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_c585512db487 (Metraipolis archive dc_c585512db487, sha256 ec81c99d7fb371502743e9008089ee7b3f273bee970bf2d9ddf138668fd9c19c)
BibTeX
@misc{dcc585512db487,
title = {Which Negatives Matter? Ask Your Text Encoder: Adaptive Similarity Margins for Dense-Caption Retrieval — arXiv:2608.18521},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.18521},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_c585512db487; sha256 ec81c99d7fb371502743e9008089ee7b3f273bee970bf2d9ddf138668fd9c19c},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.18521. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Dense-caption retrieval has recently been improved by introducing segmentation, edge maps, LLM-filtered captions, and cross-modal modules into contrastive fine-tuning. However, these methods largely inherit the same InfoNCE objective, whose optimization can prematurely saturate under a strong pre-trained initialization: on dense captions, the loss falls below 10^-3 on 80% of batches within the first epoch, while its gradient becomes numerically zero in 47% of measurements. We find that this behavior is closely related to the large number of near-duplicate captions in dense-caption benchmarks, where a few highly similar negatives remain unresolved after the easy majority has already been separated. As a remedy, we introduce HN-CLIP, which uses the text encoder's own text-text geometry to construct per-negative adaptive similarity margins. Specifically, a detached caption-similarity matrix is added to the negative logits, assigning larger margins to more similar captions without mining, synthesizing, or resampling negatives. The resulting objective requires only one caption-similarity matrix and a masked logit addition during training, with no auxiliary data, additional parameters, offline preprocessing, or inference-time overhead. Extensive experiments on four dense-caption retrieval benchmarks show that HN-CLIP improves over the strongest competitors by +2.5 to +4.0 R@1 while training 2.4x faster than GOAL and 5.4x faster than StructXLIP. Moreover, the proposed objective improves all six tested fine-tuning frameworks on the in-domain benchmarks and reaches the strongest full-data baseline with only 20% of the training data. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-20 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2608.18521 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-18T13:41:17.864Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-20?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).