Metraipolis › archive › dc_f05ba012d71a
Knowing What Not to Answer: Selective Non-Compliance in Vision-Language Models — arXiv:2609.04720
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.04720
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T22:26:35.277Z
- SHA-256 of the archived text
c5c3b1e3d6618b9858339fefb1648d0b49564fb2720cf1307df5a1c21170e25e- Stable URL of this page
https://metraipolis.com/documento/dc_f05ba012d71a
How to cite
Knowing What Not to Answer: Selective Non-Compliance in Vision-Language Models — arXiv:2609.04720 — arXiv (en), https://arxiv.org/abs/2609.04720 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f05ba012d71a (Metraipolis archive dc_f05ba012d71a, sha256 c5c3b1e3d6618b9858339fefb1648d0b49564fb2720cf1307df5a1c21170e25e)
BibTeX
@misc{dcf05ba012d71a,
title = {Knowing What Not to Answer: Selective Non-Compliance in Vision-Language Models — arXiv:2609.04720},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.04720},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f05ba012d71a; sha256 c5c3b1e3d6618b9858339fefb1648d0b49564fb2720cf1307df5a1c21170e25e},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.04720. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Vision-language models (VLMs) are expected to respond helpfully to appropriate requests while withholding compliance with requests that are incorrect, unsafe, infeasible, or unanswerable. However, existing benchmarks predominantly evaluate non-compliance at the level of the query as a whole, assuming that each request either warrants compliance or requires withholding compliance. In practice, real-world queries can contain a mixture of answerable content and components for which compliance should be withheld. In this paper, we introduce KoNA, a benchmark for evaluating selective non-compliance in VLMs across five categories: False Premise, Visual Inaccessibility, Universal Unknown, Task Feasibility, and Safety. Each task evaluates two capabilities: query-level non-compliance and component-level non-compliance under paired single and compound queries. Our evaluation across diverse VLMs shows that models often fail to refuse, correct, or abstain appropriately, and these failures become more pronounced when queries require selective non-compliance. To address this challenge, we fine-tune VLMs using KoNA examples that require selective non-compliance, together with a fully answerable set that should receive direct answers. Our fine-tuned models achieve substantial improvements in non-compliance accuracy while largely maintaining performance on fully answerable tasks. These results suggest that the fine-tuned models can distinguish between answerable components and those requiring non-compliance and respond in a task-appropriate manner. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-07 in cs.AI). Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.04720 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T22:18:11.523Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-07?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).