Metraipolis › archive › dc_afbcfe36dd08

Counterexamples as Feedback for Agent Self-Correction — arXiv:2609.02892

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.02892
License (as archived): CC0-1.0
Archived: 2026-10-17T01:11:40.160Z
SHA-256 of the archived text: 33f8e6bb3053d607c8c23d3b9fdf25c25cd57f72b485d55b222ddb4c55b83d89
Stable URL of this page: https://metraipolis.com/documento/dc_afbcfe36dd08

How to cite

Counterexamples as Feedback for Agent Self-Correction — arXiv:2609.02892 — arXiv (en), https://arxiv.org/abs/2609.02892 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_afbcfe36dd08 (Metraipolis archive dc_afbcfe36dd08, sha256 33f8e6bb3053d607c8c23d3b9fdf25c25cd57f72b485d55b222ddb4c55b83d89)

BibTeX

@misc{dcafbcfe36dd08,
  title = {Counterexamples as Feedback for Agent Self-Correction — arXiv:2609.02892},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.02892},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_afbcfe36dd08; sha256 33f8e6bb3053d607c8c23d3b9fdf25c25cd57f72b485d55b222ddb4c55b83d89},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.02892. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Single-turn code-generation metrics understate a central property of deployed agents: whether they can repair a wrong artifact after receiving concrete feedback. This paper presents A-CEGIS, a lightweight framework that uses counterexamples as feedback for evaluating multi-turn refinement in natural-language-to-regex synthesis. An agent proposes a regex, a deterministic oracle checks it under full-match semantics, and compact false-positive or false-negative witnesses guide the next turn. On 30 NL-RX-Turk tasks, diagnostic counterexample feedback solves 90\% of tasks within a four-turn ablation budget, compared with 17% for zero-shot generation, 27% for generic self-correction, and 23% for error-only feedback. In a full diagnostic run with hardening, all tasks are solved on the hidden set by the final turn, with mean time-to-success of 2.7 turns and robust success of 77% after targeted probing. These results show that A-CEGIS measures how efficiently an agent improves across turns while adding a practical robustness check beyond the original held-out cases.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-04 in cs.AI).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.02892 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T00:28:02.927Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-04?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).