Metraipolis › archive › dc_2fd27904bf42
Neologism Inoculation Prompting — arXiv:2611.04260
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.04260
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-04T18:34:29.533Z
- SHA-256 of the archived text
cf67319cc8450d6272bafc777e3f4ceda976516fe57139704113bb889073f8b4- Stable URL of this page
https://metraipolis.com/documento/dc_2fd27904bf42
How to cite
Neologism Inoculation Prompting — arXiv:2611.04260 — arXiv (en), https://arxiv.org/abs/2611.04260 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2fd27904bf42 (Metraipolis archive dc_2fd27904bf42, sha256 cf67319cc8450d6272bafc777e3f4ceda976516fe57139704113bb889073f8b4)
BibTeX
@misc{dc2fd27904bf42,
title = {Neologism Inoculation Prompting — arXiv:2611.04260},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.04260},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2fd27904bf42; sha256 cf67319cc8450d6272bafc777e3f4ceda976516fe57139704113bb889073f8b4},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.04260. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Inoculation prompting is a method to reduce undesired behaviours such as reward hacking in frontier models. It works by training a model on harmful examples under a prompt that specifies the undesired behaviour, which reduces the model's tendency to exhibit it unless prompted. However, the inoculation prompt is written in plain English, so any user can re-elicit the behaviour easily. We introduce neologism inoculation prompting (NIP), which replaces the English prompt with a new token that is trained on the frozen model and removed at deployment. NIP outperforms other inoculation methods, and performs competitively with gradient routing despite being simpler. It also works on reward hacking, but fails on sycophancy and harassment. Using control tokens and analyses of activations, we identify the mechanism for this as NIP hiding the harmful behaviour behind an unreachable region of model inputs, rather than deleting it. More broadly, our work suggests that neologisms can be a powerful way to carve out and shape access to behaviours that are undesired or harmful. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2611.04260 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-04T18:32:37.289Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).