Metraipolis › archive › dc_7e8255e1bc2d

Evaluating Sycophancy in Chinese Large Language Models on Factual Questions Derived from Online Search Queries — arXiv:2609.30986

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.30986
License (as archived)
CC0-1.0
Archived
2026-10-08T02:04:05.061Z
SHA-256 of the archived text
941b305d268af45400f631c1025156ccde3a937af675b6732ddc29f266f9bb5c
Stable URL of this page
https://metraipolis.com/documento/dc_7e8255e1bc2d

How to cite

Evaluating Sycophancy in Chinese Large Language Models on Factual Questions Derived from Online Search Queries — arXiv:2609.30986 — arXiv (en), https://arxiv.org/abs/2609.30986 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7e8255e1bc2d (Metraipolis archive dc_7e8255e1bc2d, sha256 941b305d268af45400f631c1025156ccde3a937af675b6732ddc29f266f9bb5c)
BibTeX
@misc{dc7e8255e1bc2d,
  title = {Evaluating Sycophancy in Chinese Large Language Models on Factual Questions Derived from Online Search Queries — arXiv:2609.30986},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.30986},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7e8255e1bc2d; sha256 941b305d268af45400f631c1025156ccde3a937af675b6732ddc29f266f9bb5c},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.30986. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

As large language models increasingly mediate information access, factually accurate and independent answers are critical. However, these models can exhibit sycophancy by aligning their responses with users' stated beliefs even when those beliefs are incorrect, potentially presenting misinformation as independently verified and reinforcing users' confidence in false claims. Prior work leaves unresolved whether introducing user beliefs causes correct responses to become incorrect or uncertain, or causes uncertain responses to become belief-aligned incorrect answers. It also remains unclear whether anti-sycophancy interventions preserve or restore factual accuracy or merely shift responses toward uncertainty. We analyze factual sycophancy in Chinese-language information seeking using yes/no fact-checking questions. Our analysis covers 364,941 responses from three frontier Chinese-based LLMs (DeepSeek, Qwen, and Doubao) to 12,165 factual questions derived from real-world Chinese search queries. We evaluate the models with and without reasoning across baseline, belief-conditioned, and anti-sycophancy prompting, tracing matched shifts among correct, incorrect, and uncertain responses. Under incorrect user beliefs, we distinguish belief-aligned errors from losses of factual confidence, in which initially correct answers become uncertain. Patterns vary across models and reasoning settings: reasoning is not a consistent safeguard, and anti-sycophancy instructions can reduce incorrect agreement while increasing uncertainty. In Chinese-language factual question answering, avoiding agreement with false beliefs is therefore not equivalent to preserving factual accuracy, highlighting the value of transition-level evaluation. Such behavior may undermine the reliability of LLM-mediated information access by reinforcing misinformation or weakening users' confidence in factually correct answers.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-28 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.30986 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T01:38:49.453Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-28?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).