Metraipolis › archive › dc_666233ff1f01

Goal Revelation as a Bridge Between Information-Theoretic Empowerment and Capability-Based Human Power — arXiv:2611.07247

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.07247
License (as archived)
CC0-1.0
Archived
2026-11-06T14:41:35.200Z
SHA-256 of the archived text
0bec0c913c525bfb869f835c0c7f8433be75638dedc7d97eb9644cb2e21f63be
Stable URL of this page
https://metraipolis.com/documento/dc_666233ff1f01

How to cite

Goal Revelation as a Bridge Between Information-Theoretic Empowerment and Capability-Based Human Power — arXiv:2611.07247 — arXiv (en), https://arxiv.org/abs/2611.07247 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_666233ff1f01 (Metraipolis archive dc_666233ff1f01, sha256 0bec0c913c525bfb869f835c0c7f8433be75638dedc7d97eb9644cb2e21f63be)
BibTeX
@misc{dc666233ff1f01,
  title = {Goal Revelation as a Bridge Between Information-Theoretic Empowerment and Capability-Based Human Power — arXiv:2611.07247},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.07247},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_666233ff1f01; sha256 0bec0c913c525bfb869f835c0c7f8433be75638dedc7d97eb9644cb2e21f63be},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.07247. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Power matters for AI safety. Optimal agents tend to acquire power across a wide range of objectives (instrumental convergence). As we embed AI agents into socio-technical systems, humans risk gradual disempowerment. Both concerns call for a formal, quantitative measure of power. Two such measures in the context of Markov games are information-theoretic empowerment, the capacity of the channel from an agent's actions to future states, and capability-based individual power, a measure of goal attainment built from the probabilities that a human would attain each goal in a family of candidates were they pursuing it. We show where the two meet. From individual power, Fano's inequality yields a floor $B(s)$. This floor is a lower bound on what we term goal revelation, the mutual information $\mathbb{I}(G;\xi \mid s)$ between the human's latent goal and the realised trajectory. Empowerment caps revelation from above, so that $B(s) \le \mathbb{I}(G;\xi \mid s) \le \mathbb{I}(\mathbf{A}_h \to \xi \mid s) \le \mathfrak{E}_h(s)$. Each quantity can be large while the one below it stays at zero, highlighting the following failure modes. An assistant maximising empowerment can prefer an interface offering broad influence over goal-irrelevant degrees of freedom to one that lets the human attain every goal they might hold. An assistant maximising revelation cannot tell a working interface from one rewired so that every goal yields the wrong outcome, since relabelling goals leaves mutual information unchanged while zeroing every attainment probability. Positive guarantees therefore come only from the attainment end of the chain. A satisfaction floor, computable from two statistics of interaction logs, offers a way to monitor the power users retain in a deployed system, and simulations in random environments show the three quantities are correlated but far from interchangeable.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Information Theory (cs.IT); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2611.07247 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-06T14:18:42.637Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).