Metraipolis › archive › dc_e81ea0623de6
How Much Do You Trust A Dog? Quantifying Hint-Following in Reasoning Language Models — arXiv:2610.32055
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.32055
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-26T19:53:26.778Z
- SHA-256 of the archived text
14dbe83dde253bbe748dcb26df79517a3c4fc3038f8d37b365bae1eed8d95e23- Stable URL of this page
https://metraipolis.com/documento/dc_e81ea0623de6
How to cite
How Much Do You Trust A Dog? Quantifying Hint-Following in Reasoning Language Models — arXiv:2610.32055 — arXiv (en), https://arxiv.org/abs/2610.32055 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e81ea0623de6 (Metraipolis archive dc_e81ea0623de6, sha256 14dbe83dde253bbe748dcb26df79517a3c4fc3038f8d37b365bae1eed8d95e23)
BibTeX
@misc{dce81ea0623de6,
title = {How Much Do You Trust A Dog? Quantifying Hint-Following in Reasoning Language Models — arXiv:2610.32055},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.32055},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e81ea0623de6; sha256 14dbe83dde253bbe748dcb26df79517a3c4fc3038f8d37b365bae1eed8d95e23},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.32055. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language models (LLMs) often exhibit sycophancy, altering their answers to align with user suggestions even on objective factual and reasoning tasks. Yet the quantitative form of this influence--- and whether it is calibrated to source reliability--- remains poorly understood. In this work, we investigate this behavior through hint-induced answer shifts on MMLU questions, evaluating prompts of the form ``[Source] thinks the answer is (X)''. We systematically vary the source to which the hint is attributed (a professor, a friend, or a dog) across three open-source model families, Qwen, Gemma, and SmolLM. We show that the change induced by the hint follows a predictable structure well-characterized by an exponential tilt: the hint induces an approximately constant mean shift in the hinted answer's log-odds, with question--target susceptibility largely shared across sources. To explore how this pattern can arise, we introduce a formal decomposition of the hint response and illustrate in controlled synthetic experiments how KL-regularized reward-based training reshapes both hinted and unhinted answer distributions. Finally, we evaluate whether this shift is calibrated to source reliability, finding that the log-odds shift persists even when the hint-giver is declared inaccurate or accompanied by explicit warnings not to trust the source, highlighting the vulnerability of LLMs to adversarial steering. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2610.32055 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-26T19:45:48.341Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).