Metraipolis › archive › dc_67f5726db6db
Post-Training VLMs for Video Mistake Detection — arXiv:2608.28406
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.28406
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-17T11:06:50.095Z
- SHA-256 of the archived text
1b68d110b704e74104d04f96c50470f320126da7eaf6a708932ea8a3adc3dccd- Stable URL of this page
https://metraipolis.com/documento/dc_67f5726db6db
How to cite
Post-Training VLMs for Video Mistake Detection — arXiv:2608.28406 — arXiv (en), https://arxiv.org/abs/2608.28406 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_67f5726db6db (Metraipolis archive dc_67f5726db6db, sha256 1b68d110b704e74104d04f96c50470f320126da7eaf6a708932ea8a3adc3dccd)
BibTeX
@misc{dc67f5726db6db,
title = {Post-Training VLMs for Video Mistake Detection — arXiv:2608.28406},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.28406},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_67f5726db6db; sha256 1b68d110b704e74104d04f96c50470f320126da7eaf6a708932ea8a3adc3dccd},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.28406. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Human mistakes are inevitable when following instructions, yet they can lead to severe consequences. As such, there has been an increased interest in developing methods for detecting mistakes in videos, with current methods mostly focusing on closed-set protocols. While successful in controlled settings, the closed-set assumption limits their wider applicability, as any changes to the task require collecting new data and re-training models. Instead, we argue that mistake detection methods should learn the general concept of a mistake, rather than overfitting to step-specific details. To reflect this, we introduce the Mistake Detection Video Question Answering (MD-VQA) protocol and accompanying benchmark. MD-VQA tests whether methods can discern if a step was executed correctly with respect to its description, for both seen and unseen actions. To address this important challenge, we propose the first video-language-model post-training technique for mistake detection. Our method uses a tailored reward function to encourage the model to identify discrepancies between an instruction and the corresponding video. Extensive evaluations demonstrate that this approach outperforms zero-shot, supervised fine-tuning, and post-training baselines. Notably, our method generalizes especially well to unseen procedures, for instance, with an improvement of up to 11.6% over the best-performing baseline on EP-VQA, paving the way toward general mistake detection. We release our code and benchmark at this https URL . Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-31 in cs.LG). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2608.28406 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-17T10:50:25.857Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-08-31?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).