Metraipolis › archive › dc_eaae5c5b0d73
Individual Disempowerment through an Advice Channel: Control Loss when Influence is Endogenous — arXiv:2608.14795
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.14795
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-18T17:08:07.211Z
- SHA-256 of the archived text
bfb1a98bd4a20dd75d664960242c1d84690f5b71160d993919c04401002e91ac- Stable URL of this page
https://metraipolis.com/documento/dc_eaae5c5b0d73
How to cite
Individual Disempowerment through an Advice Channel: Control Loss when Influence is Endogenous — arXiv:2608.14795 — arXiv (en), https://arxiv.org/abs/2608.14795 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_eaae5c5b0d73 (Metraipolis archive dc_eaae5c5b0d73, sha256 bfb1a98bd4a20dd75d664960242c1d84690f5b71160d993919c04401002e91ac)
BibTeX
@misc{dceaae5c5b0d73,
title = {Individual Disempowerment through an Advice Channel: Control Loss when Influence is Endogenous — arXiv:2608.14795},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.14795},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_eaae5c5b0d73; sha256 bfb1a98bd4a20dd75d664960242c1d84690f5b71160d993919c04401002e91ac},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.14795. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
An AI that can only give advice seems safe: the human is always free to ignore it. That is the premise of the boxing tradition in AI safety, and its long-suspected weak point is that the human who reads the answers is part of the system. We make the fraction $\varepsilon_t$ of behavior that follows the advice a state of a Markov decision process, moved by the advisor's own messages, so that use deepens reliance. Granted a channel rich enough to echo any action the human could take, higher $\varepsilon_t$ weakly lowers every monotone measure of the power of a human with a message-independent fallback. An oracle rewarded by per-round approval cultivates reliance beyond a closed-form patience threshold, so the same reward weights leave the optimal oracle answering in episodic deployments and cultivating in long-memory ones. An influence bound certified once at deployment is blind to that horizon and bounds the loss no lower than its trivial ceiling. An exogenous cap on influence bounds the guarantee the human loses, and a short enough memory reset removes the incentive to cultivate, while neither recovers the value already steered away. In a closed-form example the optimal oracle never cultivates in fifteen-round sessions and does in sixteen. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-18 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI) ; Computer Science and Game Theory (cs.GT). Pagina del lavoro: https://arxiv.org/abs/2608.14795 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-18T16:29:13.736Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).