Metraipolis › archive › dc_640232eeb8fa

The Self-Improvement Problem — arXiv:2610.21646

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.21646
License (as archived): CC0-1.0
Archived: 2026-10-16T09:03:51.982Z
SHA-256 of the archived text: b4a6ce30def3ddc6875bc25ead822821dc52643bbf433a421a02487f3bd7cee6
Stable URL of this page: https://metraipolis.com/documento/dc_640232eeb8fa

How to cite

The Self-Improvement Problem — arXiv:2610.21646 — arXiv (en), https://arxiv.org/abs/2610.21646 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_640232eeb8fa (Metraipolis archive dc_640232eeb8fa, sha256 b4a6ce30def3ddc6875bc25ead822821dc52643bbf433a421a02487f3bd7cee6)

BibTeX

@misc{dc640232eeb8fa,
  title = {The Self-Improvement Problem — arXiv:2610.21646},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.21646},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_640232eeb8fa; sha256 b4a6ce30def3ddc6875bc25ead822821dc52643bbf433a421a02487f3bd7cee6},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.21646. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

With the rise of large language models, agentic systems are becoming increasingly autonomous, as they write code, call tools, revise prompts, and propose modifications to their own components. This autonomy raises a fundamental question for safe self-improvement: can an automated gate like a reward model or an automated reviewer reliably decide whether a proposed update should be accepted? The difficulty is structural. The gate must be a terminating, computable procedure, while the property we ultimately care about, namely whether an update is a genuine semantic improvement, depends on potentially unbounded program behavior. We formalize this mismatch by modeling agents as programs and defining improvement in terms of a semantic score that need not be computable. We study two evaluation abstractions. The first is a certifier that approves or rejects an update. The second is a search procedure that, given access to a proposal generator, attempts to output some improving candidate if one exists. We show that, in the fully general setting, no terminating procedure can simultaneously guarantee reliability (never accepting a non-improvement) and non-conservatism (never missing an available improvement): self-referential constructions can force any such gate or searcher to fail. Randomized gates with bounded error face the same limit. We then identify a standard decidable regime used in practice (bounded evaluation and a finite proposal budget) under which exhaustive simulation yields a terminating search procedure with the desired guarantees. We also show the limit of this recovery: a chain of accepted updates can raise the bounded score in every round while the true score falls. Our results draw a boundary between unbounded, universal settings where correctness guarantees collapse and bounded settings where they can be recovered by design.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.21646 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T08:44:17.780Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).