Metraipolis › archive › dc_ca50587f4779
"You" Is Not Toxic: How LLM Labelers Rewrite the Web's Conversation Laws — arXiv:2610.23446
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.23446
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T08:44:07.342Z
- SHA-256 of the archived text
f4e23fd39026a7ab2b3541dc81361017146ca67d3860c61fc7bb0278b693e7e9- Stable URL of this page
https://metraipolis.com/documento/dc_ca50587f4779
How to cite
"You" Is Not Toxic: How LLM Labelers Rewrite the Web's Conversation Laws — arXiv:2610.23446 — arXiv (en), https://arxiv.org/abs/2610.23446 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ca50587f4779 (Metraipolis archive dc_ca50587f4779, sha256 f4e23fd39026a7ab2b3541dc81361017146ca67d3860c61fc7bb0278b693e7e9)
BibTeX
@misc{dcca50587f4779,
title = {"You" Is Not Toxic: How LLM Labelers Rewrite the Web's Conversation Laws — arXiv:2610.23446},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.23446},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ca50587f4779; sha256 f4e23fd39026a7ab2b3541dc81361017146ca67d3860c61fc7bb0278b693e7e9},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.23446. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
What the Web community believes about toxic conversations, such as that toxicity rises in longer and deeper threads, was measured mostly with Google's Perspective API, which is shutting down; large language models (LLMs) will make the next measurements. We find that LLM labelers mistake second-person address for toxicity, and that this one bias manufactures laws that human labels do not show. On 1.25 million news comments with crowd labels, Claude Haiku flags 12% of the non-toxic comments that contain no "you" and 51% of those with four or more; a single crowd rater flags 6% and 10%. Deeper replies address someone more often, so under Haiku toxicity rises with reply depth (+0.93 percentage points per level), while under the crowd's labels it slightly declines (-0.12). Rewriting "you" in the third person removes two thirds of the false rise; flagging only as many comments as the crowd does not remove it, and showing Haiku the parent comment enlarges it. All seven LLMs we test over-flag "you", and at the crowd's flag rate, without the parent, any false rise of the open ones vanishes at a fixed number of second-person words; Haiku also over-flags "you" on Reddit, Gab and Wikipedia talk pages. Agreement with human labels does not reveal the bias: we prove that a fixed Cohen's kappa leaves a law nearly unconstrained. We therefore certify the law instead of the labeler. Threadsafe uses the labeler's scores to choose which comments human raters label and to predict the others, and corrects the predictions with the human labels; linear laws are estimated without bias whatever the labeler does. In simulated audits its intervals for the laws keep nominal coverage from 250 labels on, where uniform-sample methods such as PPI++ fall short, and at 1,000 to 8,000 labels are 0.75 times as wide as those of design-based supervised learning (DSL); a thousand human labels rule out Haiku's rising law. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno). Categorie arXiv: Social and Information Networks (cs.SI) ; Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2610.23446 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-19T08:39:24.905Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).