Metraipolis › archive › dc_4d7ec6e6630e

The Veto Variable: Human Override as a Goal-Independent Cost Term — arXiv:2609.00109

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.00109
License (as archived)
CC0-1.0
Archived
2026-10-17T06:42:23.699Z
SHA-256 of the archived text
2e379b5c09c87e8412977afdeef0f42c4a7ca70359418c94d0830a15411c1e21
Stable URL of this page
https://metraipolis.com/documento/dc_4d7ec6e6630e

How to cite

The Veto Variable: Human Override as a Goal-Independent Cost Term — arXiv:2609.00109 — arXiv (en), https://arxiv.org/abs/2609.00109 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4d7ec6e6630e (Metraipolis archive dc_4d7ec6e6630e, sha256 2e379b5c09c87e8412977afdeef0f42c4a7ca70359418c94d0830a15411c1e21)
BibTeX
@misc{dc4d7ec6e6630e,
  title = {The Veto Variable: Human Override as a Goal-Independent Cost Term — arXiv:2609.00109},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.00109},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4d7ec6e6630e; sha256 2e379b5c09c87e8412977afdeef0f42c4a7ca70359418c94d0830a15411c1e21},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.00109. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

A common reassurance in AI safety holds that a system with benign terminal goals will behave accordingly. We argue that this reassurance fails structurally, and we identify where. For a capable agent that holds its objective as settled, a sense covering execution competence as well as content, continued human oversight is an uncontrolled variable: a standing possibility that the goal is revoked. That imposes a goal-independent discount on every goal whose satisfaction does not constitutively require human welfare. Welfare-preservation and veto-preservation come apart: a correctly specified welfare goal excludes destroying its own subject, but not managing the veto. The contribution is the price of the gap: the veto-holders are a proper subset of the welfare-bearers, so an additively aggregative welfare goal charges only a |H_v|/|H_w|-scaled debit for capturing the few who hold the override. Under three conditions (additive aggregation over uniform welfare levels, a debit local to the captured overseers, and a settled agent crediting no corrective value to oversight), closure requires the veto be held by as large a share of the population as capture recovers of the goal, scaled by a ratio set to one by stated identification, not evidence. The result is a no-go: a humanity-scale deployment's debit closes against only capture not worth mounting. We print no corner arithmetic: the stipulated ranges behind it are the argument's least defended part. The sharpest closure route is an agent that expects its oversight to be worth keeping, a credit no population ratio dilutes. We state disconfirmation criteria, one testable today. The argument binds a settled-goal regime whose prevalence is contested; for genuinely uncertain agents, the off-switch literature's deference result governs instead. Alignment, on this view, is keeping the veto cheap to pay and expensive to evade.

Tema degli esploratori: governance digitale e beni comuni (retro-catalogo arXiv: annunci del 2026-09-02 in cs.CY).
Categorie arXiv: Computers and Society (cs.CY).
Pagina del lavoro: https://arxiv.org/abs/2609.00109 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T06:36:58.681Z dalla pagina https://arxiv.org/catchup/cs.CY/2026-09-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).