Metraipolis › archive › dc_196dda4fe4e1

Dr. No: Feature Steering for Abstention in Vision-Language Models — arXiv:2611.03862

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.03862
License (as archived): CC0-1.0
Archived: 2026-11-04T12:39:20.912Z
SHA-256 of the archived text: fb674d7814a0d196370a3b8b574b25779228ccd864140b7506112f97193da433
Stable URL of this page: https://metraipolis.com/documento/dc_196dda4fe4e1

How to cite

Dr. No: Feature Steering for Abstention in Vision-Language Models — arXiv:2611.03862 — arXiv (en), https://arxiv.org/abs/2611.03862 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_196dda4fe4e1 (Metraipolis archive dc_196dda4fe4e1, sha256 fb674d7814a0d196370a3b8b574b25779228ccd864140b7506112f97193da433)

BibTeX

@misc{dc196dda4fe4e1,
  title = {Dr. No: Feature Steering for Abstention in Vision-Language Models — arXiv:2611.03862},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.03862},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_196dda4fe4e1; sha256 fb674d7814a0d196370a3b8b574b25779228ccd864140b7506112f97193da433},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.03862. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large foundation models often answer even when the available visual or textual evidence is insufficient to support a reliable conclusion. This failure to abstain makes them hard to trust in high-stakes or information-seeking settings, where a confident but unsupported answer can be worse than no answer at all. It also raises a natural question: can the internal mechanisms behind this behaviour be identified and controlled at inference time? We present Dr. No, a feature-steering approach that mines sparse autoencoder (SAE) features associated with abstention and commitment, then modulates a small set of them during the prefill pass, without retraining the base model. Dr. No is evaluated on LLaVA-1.6 Mistral 7B across multimodal abstention benchmarks, including UNK-VQA and MM-UPD, and additionally benchmarked on language reasoning tasks using Qwen3.5-2B on AbstentionBench subsets. Against multiple uncertainty-based abstention baselines, feature steering achieves a more favourable balance between correct answering and appropriate abstention than post-hoc rejection methods. The gains are strongest on UNK-VQA and on MM-UPD cases involving image--question or question--answer incompatibility, and transfer most clearly to mathematical reasoning subsets of AbstentionBench. However, performance is more limited on absent-answer detection and retrieval-heavy multi-hop QA, such as MuSiQue. These results suggest that sparse internal features provide lightweight control handles for shaping uncertainty-aware behaviour in foundation models, while also showing that abstention control is source-dependent rather than governed by a single universal signal.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2611.03862 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-04T12:29:47.249Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).