Metraipolis › archive › dc_f50bcb6b9b4e
Queries Knew More Than We Thought: Uncovering Latent Knowledge in Segmentation Models — arXiv:2609.20283
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.20283
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-10T05:07:54.161Z
- SHA-256 of the archived text
58eb9288d7fd874d2ee8f32bf1d212f49b030c9cf25f361e2f0d6d0acf6f32ee- Stable URL of this page
https://metraipolis.com/documento/dc_f50bcb6b9b4e
How to cite
Queries Knew More Than We Thought: Uncovering Latent Knowledge in Segmentation Models — arXiv:2609.20283 — arXiv (en), https://arxiv.org/abs/2609.20283 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f50bcb6b9b4e (Metraipolis archive dc_f50bcb6b9b4e, sha256 58eb9288d7fd874d2ee8f32bf1d212f49b030c9cf25f361e2f0d6d0acf6f32ee)
BibTeX
@misc{dcf50bcb6b9b4e,
title = {Queries Knew More Than We Thought: Uncovering Latent Knowledge in Segmentation Models — arXiv:2609.20283},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.20283},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f50bcb6b9b4e; sha256 58eb9288d7fd874d2ee8f32bf1d212f49b030c9cf25f361e2f0d6d0acf6f32ee},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.20283. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Modern segmenters often fail after the expensive computation has already been done: a useful mask is present among the model's query-conditioned candidates, but the deployed selection rule does not expose it. We study this output-selection bottleneck in frozen DETR-family models. A ground-truth-only oracle first shows substantial hidden headroom in already-computed mask proposals. This raises a simple question: How can we better use the masks a segmenter has already computed but does not expose? We then ask whether that headroom can be recovered without adding queries, generating new masks, rerunning the backbone, or updating weights. HYDRA is a small selector trained only on cached frozen outputs. At inference time, it scores the cached candidates against an explicit keep-baseline option and acts only when a held-out calibrated margin indicates the selected candidate is sufficiently better. Trained on training-split caches and calibrated on held-out data, HYDRA improves Mask2Former, MaskDINO, and OneFormer by up to +7.41 dataset mIoU points on ADE20k and COCO, and improves SAM 3 by +9.4 class-macro prompt-IoU points on average across eight domains while preserving useful predictions through calibration. Paired LoRA controls show that lightweight weight adaptation does not remove the bottleneck: exposed predictions are often flat or worse, while routing over the adapted candidates still recovers accuracy. Finally, we connect the effect to query specialization under bipartite matching and verify it in a controlled TinyDETR study. These results show that frozen segmenters should be evaluated not only by the masks they expose, but also by the useful candidates they suppress. Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-18 in cs.CV). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV). Pagina del lavoro: https://arxiv.org/abs/2609.20283 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-10T04:26:09.596Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-09-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).