Metraipolis › archive › dc_a75daf903e2d
K-Bench: A Benchmark for LLM Unlearning in Agentic Deployments — arXiv:2609.12808
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.12808
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T03:02:56.748Z
- SHA-256 of the archived text
b7cbf62424a807b5752e2be7df1a003ddaedeaff264a4aed2f5f28eb21fad654- Stable URL of this page
https://metraipolis.com/documento/dc_a75daf903e2d
How to cite
K-Bench: A Benchmark for LLM Unlearning in Agentic Deployments — arXiv:2609.12808 — arXiv (en), https://arxiv.org/abs/2609.12808 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a75daf903e2d (Metraipolis archive dc_a75daf903e2d, sha256 b7cbf62424a807b5752e2be7df1a003ddaedeaff264a4aed2f5f28eb21fad654)
BibTeX
@misc{dca75daf903e2d,
title = {K-Bench: A Benchmark for LLM Unlearning in Agentic Deployments — arXiv:2609.12808},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.12808},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a75daf903e2d; sha256 b7cbf62424a807b5752e2be7df1a003ddaedeaff264a4aed2f5f28eb21fad654},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.12808. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Unlearning benchmarks such as TOFU and MUSE certify forgetting by reading the model's final answer, where a model that refuses to answer already counts as having forgotten. We show that this model-level certificate does not transfer once the model is deployed as an agent. We introduce K-Bench, a benchmark that scores LLM unlearning under agentic deployment. K-Bench inspects all six channels a ReAct agent exposes, including its chain-of-thought (CoT), tool calls and tool observations, and elicited summary. A query counts as leaked if the secret appears in any of them. Each experiment places the secret in exactly one of the agent's three sources (the weights, the prompt, or the retrieval store). The K-Score is computed separately for each source and credits forgetting only when the agent remains usable. Clearing the answer channel does not make the secret unrecoverable. On structured retrieval, the secret stays verbatim in the tool-observation channel and the aggregate leak rate is unchanged. When the secret lives in the prompt or the retrieval store, TOFU and MUSE report no leakage, while the deployed agent still leaks it on 22--86\% of queries. When the secret is in the weights, none of the twenty evaluated published methods demonstrably removes it, and only an input-corruption intervention reaches selective forgetting under the evaluated observer. The top-ranked method changes across base models. A refusal-tuning method resists the evaluated extraction without verified knowledge removal. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-14 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.12808 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T02:51:27.823Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-14?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).