Metraipolis › archive › dc_b2d45c992cf2

Build it, Break it, Repeat: Benchmarking and improving LLM-manipulated disinformation detection in social media posts — arXiv:2608.09510

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.09510
License (as archived)
CC0-1.0
Archived
2026-10-19T08:50:24.204Z
SHA-256 of the archived text
edb4fa51dc99c0a9eef60897831956d58cd7c217a1486e04eb27dbc15d5aa2e4
Stable URL of this page
https://metraipolis.com/documento/dc_b2d45c992cf2

How to cite

Build it, Break it, Repeat: Benchmarking and improving LLM-manipulated disinformation detection in social media posts — arXiv:2608.09510 — arXiv (en), https://arxiv.org/abs/2608.09510 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b2d45c992cf2 (Metraipolis archive dc_b2d45c992cf2, sha256 edb4fa51dc99c0a9eef60897831956d58cd7c217a1486e04eb27dbc15d5aa2e4)
BibTeX
@misc{dcb2d45c992cf2,
  title = {Build it, Break it, Repeat: Benchmarking and improving LLM-manipulated disinformation detection in social media posts — arXiv:2608.09510},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.09510},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b2d45c992cf2; sha256 edb4fa51dc99c0a9eef60897831956d58cd7c217a1486e04eb27dbc15d5aa2e4},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.09510. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Detecting machine-generated disinformation on social media is increasingly difficult as large language models (LLMs) make it easier to generate and rewrite misleading content at scale. Static benchmark evaluations, measuring detector performance on fixed held-out datasets, do not capture how detectors behave when posts are deliberately transformed to evade classification. This paper adapts the Build it, Break it, Fix it framework into Build it, Break it, Repeat (BiBiR): iterative sessions designed to stress-test detectors' robustness under iterative adversarial conditions, evaluating whether models remain reliable when disinformation posts are systematically transformed to evade classification. Across five iterations, the findings show that the best adversarial breakers' transformations came from a combination of back-translation and LLM persona-based rewriting, with the best performing technique achieving a 95% label flip rate (LFR), whilst still preserving the meaning of the original posts. The best builders' model was a triplet contrastive model with a dynamic anchor switching (DASS) architecture, which achieved an average accuracy of 72.68%, outperforming the strong baseline (a fine-tuned e5-small-LoRA) by 15 percentage points on the most robust set of breakers' adversarial attacks. The results demonstrate that an iterative framework best exposes detector weaknesses and pushes robustness improvements; however, it may still require semantic preservation analysis to distinguish valid adversarial evasion from transformations that changed the original disinformation claims' meaning.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-11 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Social and Information Networks (cs.SI).
Pagina del lavoro: https://arxiv.org/abs/2608.09510 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T08:49:41.931Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).