Metraipolis › archive › dc_ea3e1d444431

No Note Is an Island: Probing LLMs' Understanding of Crowdsourced Fact-Checking Environments — arXiv:2611.01935

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.01935
License (as archived)
CC0-1.0
Archived
2026-11-03T18:31:27.876Z
SHA-256 of the archived text
b3671b70c58094414b31705625ee38f5677f334e93c072ad6e6c2ac14cb98bb6
Stable URL of this page
https://metraipolis.com/documento/dc_ea3e1d444431

How to cite

No Note Is an Island: Probing LLMs' Understanding of Crowdsourced Fact-Checking Environments — arXiv:2611.01935 — arXiv (en), https://arxiv.org/abs/2611.01935 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ea3e1d444431 (Metraipolis archive dc_ea3e1d444431, sha256 b3671b70c58094414b31705625ee38f5677f334e93c072ad6e6c2ac14cb98bb6)
BibTeX
@misc{dcea3e1d444431,
  title = {No Note Is an Island: Probing LLMs' Understanding of Crowdsourced Fact-Checking Environments — arXiv:2611.01935},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.01935},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ea3e1d444431; sha256 b3671b70c58094414b31705625ee38f5677f334e93c072ad6e6c2ac14cb98bb6},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.01935. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language models (LLMs) increasingly assist crowdsourced fact-checking, yet evaluations focus on individual annotations. A correction's reach and reception depend on visibility, audience responses, and platform policies. We introduce correction context understanding: assessing corrective content, interpreting audience responses, and reasoning about platform dynamics within a shared environment. We build NoteHabitat from 36 months of Weibo Annotations data, linking 102,994 posts (42,853 annotations), 431,407 comments, and five policy events. Evaluating diverse model families across seven tasks spanning content, interaction, and management reveals three gaps. Annotation quality correlates only weakly with engagement ($\rho=0.068$), while four zero-shot LLMs systematically underrate quality relative to human judgments, especially evidence support and objectivity. Models also miss toxic comments supporting corrections (36.4% average recall across ten models). At the platform level, LLMs partially predict policy-associated outcome directions, but LLM-guided simulations poorly reproduce daily activity. Cross-group analyses expose blind spots tied to contribution types and reader stances that aggregate scores conceal. Reliable fact-checking AI must understand the note, the conversation around it, and the platform in which both unfold.

Tema degli esploratori: governance digitale e beni comuni (elenco arXiv cs.SI, nuovi lavori del giorno).
Categorie arXiv: Social and Information Networks (cs.SI).
Pagina del lavoro: https://arxiv.org/abs/2611.01935 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T18:12:07.784Z dalla pagina https://arxiv.org/list/cs.SI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).