Metraipolis › archive › dc_aaed2724bed4
No Note Is an Island: Probing LLMs' Understanding of Crowdsourced Fact-Checking Environments — arXiv:2611.01935
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.01935
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T11:16:52.783Z
- SHA-256 of the archived text
e17acfcc3a699c63af7c406ebf1e647036a17673c0e32da67441d1d19098ee6e- Stable URL of this page
https://metraipolis.com/documento/dc_aaed2724bed4
How to cite
No Note Is an Island: Probing LLMs' Understanding of Crowdsourced Fact-Checking Environments — arXiv:2611.01935 — arXiv (en), https://arxiv.org/abs/2611.01935 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_aaed2724bed4 (Metraipolis archive dc_aaed2724bed4, sha256 e17acfcc3a699c63af7c406ebf1e647036a17673c0e32da67441d1d19098ee6e)
BibTeX
@misc{dcaaed2724bed4,
title = {No Note Is an Island: Probing LLMs' Understanding of Crowdsourced Fact-Checking Environments — arXiv:2611.01935},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.01935},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_aaed2724bed4; sha256 e17acfcc3a699c63af7c406ebf1e647036a17673c0e32da67441d1d19098ee6e},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.01935. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language models (LLMs) increasingly assist crowdsourced fact-checking, yet evaluations focus on individual annotations. A correction's reach and reception depend on visibility, audience responses, and platform policies. We introduce correction context understanding: assessing corrective content, interpreting audience responses, and reasoning about platform dynamics within a shared environment. We build NoteHabitat from 36 months of Weibo Annotations data, linking 102,994 posts (42,853 annotations), 431,407 comments, and five policy events. Evaluating diverse model families across seven tasks spanning content, interaction, and management reveals three gaps. Annotation quality correlates only weakly with engagement ($\rho=0.068$), while four zero-shot LLMs systematically underrate quality relative to human judgments, especially evidence support and objectivity. Models also miss toxic comments supporting corrections (36.4% average recall across ten models). At the platform level, LLMs partially predict policy-associated outcome directions, but LLM-guided simulations poorly reproduce daily activity. Cross-group analyses expose blind spots tied to contribution types and reader stances that aggregate scores conceal. Reliable fact-checking AI must understand the note, the conversation around it, and the platform in which both unfold. Tema degli esploratori: governance digitale e beni comuni (elenco arXiv cs.SI, nuovi lavori del giorno). Categorie arXiv: Social and Information Networks (cs.SI). Pagina del lavoro: https://arxiv.org/abs/2611.01935 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-03T11:01:52.143Z dalla pagina https://arxiv.org/list/cs.SI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).