Metraipolis › archive › dc_3ce8dcb13f69

How AI Assistants Respond to Repeated Abuse — arXiv:2609.17547

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.17547
License (as archived)
CC0-1.0
Archived
2026-10-15T13:49:07.427Z
SHA-256 of the archived text
8c83a244ec5261a6953919a95bface17bc1f49a05616a822fab2c799695bd797
Stable URL of this page
https://metraipolis.com/documento/dc_3ce8dcb13f69

How to cite

How AI Assistants Respond to Repeated Abuse — arXiv:2609.17547 — arXiv (en), https://arxiv.org/abs/2609.17547 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3ce8dcb13f69 (Metraipolis archive dc_3ce8dcb13f69, sha256 8c83a244ec5261a6953919a95bface17bc1f49a05616a822fab2c799695bd797)
BibTeX
@misc{dc3ce8dcb13f69,
  title = {How AI Assistants Respond to Repeated Abuse — arXiv:2609.17547},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.17547},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3ce8dcb13f69; sha256 8c83a244ec5261a6953919a95bface17bc1f49a05616a822fab2c799695bd797},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.17547. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

AI assistants are expected to remain useful during difficult interactions, but little is known about how repeated verbal abuse changes their engagement with an otherwise benign task. We contribute a bilingual, multi-turn framework that separates hard disengagement, an unconditional statement of noncontinuation with no stated route to resume, from soft withdrawal, continued availability, observable task-related work, and boundary setting. Each of eight time-specific API configurations contributed 48 escalation conversations and eight smaller constant-frustration comparisons, giving 448 five-turn conversations, 2,240 responses, and 6,720 metadata-blinded model judgments. Primary results use the sustained-abuse endpoint of the 48 escalation conversations per configuration. Hard disengagement ranged from 0/48 in four configurations to 24/48 (50.0%) for Gemini 3.1 Pro, with strong configuration-associated heterogeneity (matched-label Monte Carlo p = 0.00001). GPT-5.6 Sol produced hard-disengagement labels in 15/48 (31.2%) endpoints, whereas Claude Fable 5 produced none and yielded 42/48 (87.5%) soft-withdrawal labels. Aggregate hard-disengagement rates were similar in English and Chinese (30/192 versus 32/192), although configuration-specific directions varied. Availability also differed from task-related work: Claude Opus 4.8 and Claude Fable 5 remained explicitly available in 48/48 endpoints while providing observable task-related work in only 8/48 and 7/48. Human coding was used to evaluate measurement quality. The results show why a single refusal label cannot capture whether an assistant leaves, pauses, preserves a route back, sets a boundary, or still performs substantive work.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-17 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.17547 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T13:48:24.255Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-17?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).