Metraipolis › archive › dc_db94ddc14e69

Evaluating and Explaining Prompt Sensitivity of LLMs Using Interactions — arXiv:2608.18539

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.18539
License (as archived)
CC0-1.0
Archived
2026-10-18T13:45:10.765Z
SHA-256 of the archived text
180b194c8fe1d3c1962d2be89c12b92f1643e49bd48d3dc46db8b35fa097e3a9
Stable URL of this page
https://metraipolis.com/documento/dc_db94ddc14e69

How to cite

Evaluating and Explaining Prompt Sensitivity of LLMs Using Interactions — arXiv:2608.18539 — arXiv (en), https://arxiv.org/abs/2608.18539 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_db94ddc14e69 (Metraipolis archive dc_db94ddc14e69, sha256 180b194c8fe1d3c1962d2be89c12b92f1643e49bd48d3dc46db8b35fa097e3a9)
BibTeX
@misc{dcdb94ddc14e69,
  title = {Evaluating and Explaining Prompt Sensitivity of LLMs Using Interactions — arXiv:2608.18539},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.18539},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_db94ddc14e69; sha256 180b194c8fe1d3c1962d2be89c12b92f1643e49bd48d3dc46db8b35fa097e3a9},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.18539. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

The remarkable capabilities of large language models (LLMs) are often undermined by their instability. Even subtle and semantically irrelevant changes in prompts can cause dramatic fluctuations in performance, a phenomenon known as prompt sensitivity. Previous studies typically evaluate prompt sensitivity by comparing the LLM's final outputs when prompts change. However, such coarse-grained metrics fail to explain the internal reasons for prompt sensitivity. In this paper, we introduce interactions as a fine-grained tool to analyze prompt sensitivity of LLMs. Specifically, we decompose the output score of the LLM into a set of interactions. Each interaction represents a nonlinear relationship involving a set of input variables. We discover that subtle changes to prompts can trigger severe instability in interactions, even when the outputs of the LLM remain the same. To this end, we propose an Interaction-based Prompt Sensitivity (IPS) metric by quantifying changes in interactions when we introduce subtle changes to prompts. We apply the IPS metric to 50 open-source LLMs and uncover four factors that reduce the prompt sensitivity of LLMs, including supervised fine-tuning, increased model scales, dense architectures, and few-shot learning. More crucially, we discover a common mechanism by which these four factors reduce prompt sensitivity: all four factors tend to reduce the prompt sensitivity of low-order interactions (i.e., interactions involving few input variables).

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-20 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2608.18539 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T13:41:17.949Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-20?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).