Metraipolis › archive › dc_3de5ba0d97bc
Neologism Inoculation Prompting — arXiv:2611.04260
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.04260
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-04T12:27:45.833Z
- SHA-256 of the archived text
ea93d54eb21fcb39937582f9a6c1a59e4ab6a760fc88d9acf54a4a992f3298ac- Stable URL of this page
https://metraipolis.com/documento/dc_3de5ba0d97bc
How to cite
Neologism Inoculation Prompting — arXiv:2611.04260 — arXiv (en), https://arxiv.org/abs/2611.04260 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3de5ba0d97bc (Metraipolis archive dc_3de5ba0d97bc, sha256 ea93d54eb21fcb39937582f9a6c1a59e4ab6a760fc88d9acf54a4a992f3298ac)
BibTeX
@misc{dc3de5ba0d97bc,
title = {Neologism Inoculation Prompting — arXiv:2611.04260},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.04260},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3de5ba0d97bc; sha256 ea93d54eb21fcb39937582f9a6c1a59e4ab6a760fc88d9acf54a4a992f3298ac},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.04260. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Inoculation prompting is a method to reduce undesired behaviours such as reward hacking in frontier models. It works by training a model on harmful examples under a prompt that specifies the undesired behaviour, which reduces the model's tendency to exhibit it unless prompted. However, the inoculation prompt is written in plain English, so any user can re-elicit the behaviour easily. We introduce neologism inoculation prompting (NIP), which replaces the English prompt with a new token that is trained on the frozen model and removed at deployment. NIP outperforms other inoculation methods, and performs competitively with gradient routing despite being simpler. It also works on reward hacking, but fails on sycophancy and harassment. Using control tokens and analyses of activations, we identify the mechanism for this as NIP hiding the harmful behaviour behind an unreachable region of model inputs, rather than deleting it. More broadly, our work suggests that neologisms can be a powerful way to carve out and shape access to behaviours that are undesired or harmful. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2611.04260 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-04T12:26:47.138Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).