Metraipolis › archive › dc_1604c68153bf
Counterfactual Test-Time Augmentation for Shortcut Removal in Language Models — arXiv:2610.33904
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.33904
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-27T08:00:33.609Z
- SHA-256 of the archived text
b7df4d28abc32f30daeadf750c006473f1763d012c39b704eb8f408cf01fa99d- Stable URL of this page
https://metraipolis.com/documento/dc_1604c68153bf
How to cite
Counterfactual Test-Time Augmentation for Shortcut Removal in Language Models — arXiv:2610.33904 — arXiv (en), https://arxiv.org/abs/2610.33904 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_1604c68153bf (Metraipolis archive dc_1604c68153bf, sha256 b7df4d28abc32f30daeadf750c006473f1763d012c39b704eb8f408cf01fa99d)
BibTeX
@misc{dc1604c68153bf,
title = {Counterfactual Test-Time Augmentation for Shortcut Removal in Language Models — arXiv:2610.33904},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.33904},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_1604c68153bf; sha256 b7df4d28abc32f30daeadf750c006473f1763d012c39b704eb8f408cf01fa99d},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.33904. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language models (LLMs) can exploit shortcuts, i.e., correlations present in training data that may not hold at test time. This can lead to poor predictions when correlations shift. Most existing solutions for removing shortcuts operate at training time, requiring labeled training data, model weight access, and significant computational resources. Here, we study the problem of test-time shortcut removal. We propose a family of counterfactual test-time augmentation (CTTA) methods: given an example, each method uses an auxiliary model to generate counterfactual edits in which the shortcut value is changed, and then takes a weighted average of the LLM's predictions across these edits. We prove that the optimal shortcut-invariant predictor is a member of this CTTA family, and we characterize its weights. The weights depend on the training distribution, and therefore are, in general, not identifiable from the trained model and unlabeled test data. However, we present assumptions under which the weights can be obtained using the token probabilities of the trained LLM, and propose tests to detect violations of these assumptions. We use these tests to develop Select-CTTA, an aggregate method that selects among CTTA methods. Empirically, we find that Select-CTTA outperforms test-time baselines and approaches the performance of models fine-tuned on shortcut-balanced data. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2610.33904 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-27T07:55:30.862Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).