Metraipolis › archive › dc_bcac49945258

LLM Detoxification via Transition-Aware Reward-Guided Representation Sampling — arXiv:2610.41165

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.41165
License (as archived)
CC0-1.0
Archived
2026-11-02T04:43:04.102Z
SHA-256 of the archived text
27ab1e00b14f4547e93ab4221835ff1a9a13eb2a89666dbf3fa34e9eeac3248e
Stable URL of this page
https://metraipolis.com/documento/dc_bcac49945258

How to cite

LLM Detoxification via Transition-Aware Reward-Guided Representation Sampling — arXiv:2610.41165 — arXiv (en), https://arxiv.org/abs/2610.41165 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bcac49945258 (Metraipolis archive dc_bcac49945258, sha256 27ab1e00b14f4547e93ab4221835ff1a9a13eb2a89666dbf3fa34e9eeac3248e)
BibTeX
@misc{dcbcac49945258,
  title = {LLM Detoxification via Transition-Aware Reward-Guided Representation Sampling — arXiv:2610.41165},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.41165},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bcac49945258; sha256 27ab1e00b14f4547e93ab4221835ff1a9a13eb2a89666dbf3fa34e9eeac3248e},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.41165. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large Language Models (LLMs) can generate toxic content, motivating effective test-time detoxification. Existing representation steering methods often rely on static or coarse interventions and provide imprecise guidance because they inadequately model transitions between toxic and non-toxic generations. We propose Transition-Aware Reward-Guided Representation Sampling (TARGRS), a test-time framework that learns reward guidance over toxicity transitions in latent representation space. During exploration, progressive steering along the toxicity direction constructs trajectories from non-toxic to toxic states. Toxicity scores along these trajectories turn sparse annotations into dense, listwise supervision for a transition-aware token-level reward model. During inference, TARGRS uses this model to guide coarse-to-fine sampling along the opposite, non-toxic direction: coarse sampling identifies the intervention needed to reach non-toxic regions, and refinement selects a detoxified representation with minimal perturbation. Experiments on 10 LLMs across RealToxicityPrompts, BOLD, and AttaQ show that TARGRS consistently outperforms state-of-the-art baselines, reducing toxicity by up to 69.38% on RealToxicityPrompts. TARGRS runs nearly as fast as the original LLM while preserving its core capabilities with minimal impact. By intervening in the language decoder space, TARGRS also extends to Large Vision-Language Models (LVLMs), reducing toxicity by up to 64.23% across 3 representative LVLMs.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.41165 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-02T04:42:12.352Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).