Metraipolis › archive › dc_dc23734927f7
To Jev or Not? Evaluating the Accuracy and Efficiency of Structured Decision Models for Hate-Speech Moderation — arXiv:2610.03324
- Source
- arXiv
- Original
- https://arxiv.org/abs/2610.03324
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-05T22:51:47.089Z
- SHA-256 of the archived text
c913d879720e76afac812c370c35beb263d1e73a23ffe1539144636ec00ad0ad- Stable URL of this page
https://metraipolis.com/documento/dc_dc23734927f7
How to cite
To Jev or Not? Evaluating the Accuracy and Efficiency of Structured Decision Models for Hate-Speech Moderation — arXiv:2610.03324 — arXiv, https://arxiv.org/abs/2610.03324 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_dc23734927f7 (Metraipolis archive dc_dc23734927f7, sha256 c913d879720e76afac812c370c35beb263d1e73a23ffe1539144636ec00ad0ad)
BibTeX
@misc{dcdc23734927f7,
title = {To Jev or Not? Evaluating the Accuracy and Efficiency of Structured Decision Models for Hate-Speech Moderation — arXiv:2610.03324},
howpublished = {arXiv},
url = {https://arxiv.org/abs/2610.03324},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_dc23734927f7; sha256 c913d879720e76afac812c370c35beb263d1e73a23ffe1539144636ec00ad0ad},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.03324. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
The scale of online content makes hate-speech moderation challenging, while Large Language Models (LLMs) enable harmful material to be produced and adapted more easily. Moderation therefore requires efficient classifiers that can accommodate different definitions of hate speech. Recent structured decision models accept natural-language criteria and select among specified answers, raising the question of whether they can meet these requirements without task-specific training. We present HATEDECIDE, an evaluation of six decision-model configurations on four hate-speech datasets against specialized moderation, zero-shot, commercial, and supervised baselines. We examine whether supplying a dataset's definition, or decomposing it into multiple questions, improves classification, and we measure their latency and cost. We find that commercial LLMs significantly outperform all decision models on only one dataset. Supplying definitions changes up to 28\% of predictions without consistently improving classification, and decomposition significantly improves performance in only 20\% of the comparisons. On a diagnostic set of test cases, the best hosted decision model comes within 1.6 macro-F1 points of the best commercial LLM at approximately 97\% lower inference cost. These results identify opportunities for inexpensive moderation, while showing that explicit criteria and additional questions do not reliably improve classification. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2610.03324 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-05T22:46:10.584Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).