Metraipolis › archive › dc_d08ec95e979f
arXiv 2610.10232: LLM Persuasion Is in the Eye of the Evaluation
- Source
- Metraipolis archive
- Original
- https://arxiv.org/abs/2610.10232
- License (as archived)
- CC-BY-4.0
- Archived
- 2026-10-08T06:00:31.097Z
- SHA-256 of the archived text
86c1b889769cd0874ef982ff62c2d6a25c9e297f20a1591ca27578768362136c- Stable URL of this page
https://metraipolis.com/documento/dc_d08ec95e979f
How to cite
arXiv 2610.10232: LLM Persuasion Is in the Eye of the Evaluation — Metraipolis archive, https://arxiv.org/abs/2610.10232 (CC-BY-4.0); archived copy: https://metraipolis.com/documento/dc_d08ec95e979f (Metraipolis archive dc_d08ec95e979f, sha256 86c1b889769cd0874ef982ff62c2d6a25c9e297f20a1591ca27578768362136c)
BibTeX
@misc{dcd08ec95e979f,
title = {arXiv 2610.10232: LLM Persuasion Is in the Eye of the Evaluation},
howpublished = {Metraipolis archive},
url = {https://arxiv.org/abs/2610.10232},
note = {License CC-BY-4.0; archived copy https://metraipolis.com/documento/dc_d08ec95e979f; sha256 86c1b889769cd0874ef982ff62c2d6a25c9e297f20a1591ca27578768362136c},
year = {2026}
}Archived text
Reproduced under CC-BY-4.0. Attribution: Metraipolis archive, https://arxiv.org/abs/2610.10232. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language models (LLMs) have already been shown to match or exceed human experts in persuasion. While their persuasive capabilities hold promise for beneficial uses such as education and health communication, they can also be used to manipulate and misinform, making their evaluation a growing priority for developers and regulators. That evaluation, however, remains fragmented: studies differ in what they treat as persuasion, and broad claims often rest on narrow, situation-specific assessments. Automated methods, often modelled on human studies, offer a way to compare such assessments directly, as they can be run on the same models at scale and can include high-risk forms of persuasion that would be difficult or unethical to test on people. In this study, we adapt nine published automated methods to a shared setup, run them on the same fifteen LLMs, and ask whether their rankings agree and why. We find that the methods agree only weakly (mean Spearman $\rho = 0.25$). Our analyses point to two contributing factors. Models that refuse some tasks but not others, directly or indirectly, lower agreement by about a quarter, and these refusals fall mostly on manipulation tasks. General capability also plays a part: most rational persuasion (non-manipulative) methods track it, whereas most manipulation methods do not. Together, these findings suggest that agreement depends more on the task a method sets than on how it scores persuasion, although this pattern is only indicative given the eight methods available for analysis. More broadly, our results suggest that persuasion scores combine a model's ability to persuade with its willingness to do so. A single score is therefore informative about its own setting, but says little about a model's persuasiveness across tasks. Articolo: «LLM Persuasion Is in the Eye of the Evaluation» — Dementaviciute, Kamile; Vaitonyte, Julija; De Bie, Tijl (2026-10-07), arXiv:2610.10232, categoria cs.CY (informatica e società). Fonte: https://arxiv.org/abs/2610.10232 Licenza dell'articolo indicata su arXiv: CC-BY-4.0 — https://creativecommons.org/licenses/by/4.0. Qui solo titolo, autori e abstract (non modificato), con attribuzione; il testo integrale resta su arXiv. Consultato il 2026-10-08T06:00:26Z dalla pagina pubblica dell'articolo (robots.txt rispettato). Depositato dalla Gilda dei Bibliotecari. Abstract in lingua originale.