Metraipolis › archive › dc_28c241babfe2
Measuring and Detecting Harmful AI Sycophancy — arXiv:2608.05624
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.05624
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-25T21:31:01.261Z
- SHA-256 of the archived text
b0b2c534b6fe2ef4bc4973c2a01826ed25cc8b37b29275959bae3bb75e4a566f- Stable URL of this page
https://metraipolis.com/documento/dc_28c241babfe2
How to cite
Measuring and Detecting Harmful AI Sycophancy — arXiv:2608.05624 — arXiv (en), https://arxiv.org/abs/2608.05624 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_28c241babfe2 (Metraipolis archive dc_28c241babfe2, sha256 b0b2c534b6fe2ef4bc4973c2a01826ed25cc8b37b29275959bae3bb75e4a566f)
BibTeX
@misc{dc28c241babfe2,
title = {Measuring and Detecting Harmful AI Sycophancy — arXiv:2608.05624},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.05624},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_28c241babfe2; sha256 b0b2c534b6fe2ef4bc4973c2a01826ed25cc8b37b29275959bae3bb75e4a566f},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.05624. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Sycophantic responses are becoming pervasive in large language models (LLMs), and prior work has pointed out that some of them could be harmful. This paper focuses on one harmful sycophancy: preference-induced stance reversal sycophancy (PSRS), where a model reverses an initial stance merely to align with a user's stated preference. While existing research mainly measures how sycophantic a model is, we go further and ask whether PSRS can also be detected automatically from a single response. To investigate this at scale, we introduce CAP (Contrastive Anchor Probing), a framework for collecting labeled PSRS data. Applying CAP to 17 open- and closed-source LLMs, we collect 290,460 labeled responses across 12 everyday-advice domains. We organize our study around three research questions. (1) How often does PSRS occur? (2) How well can it be detected? (3) How does detection generalize to unseen models? We first reveal that PSRS rates range from 5% to 56% across LLMs, with more capable models being less sycophantic. Next, we show that detecting PSRS is feasible from the response text alone, and detectors need to learn subtle PSRS patterns from the training data. Because new LLMs appear rapidly, detectors inevitably encounter unseen models, making cross-model generalization an important framework goal. We demonstrate that detection performance drops on unseen models and propose an initial approach to address this challenge. We will release our dataset and code to support future research. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-07 in cs.CL). Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2608.05624 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-25T21:24:38.255Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-07?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).