Metraipolis › archive › dc_ca3d36a6845a

DMAIC-Bench: From Self-Reflection to Self-Restraint for Agents in Multi-Stage Analytical Tasks — arXiv:2610.32305

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.32305
License (as archived)
CC0-1.0
Archived
2026-10-26T19:50:33.489Z
SHA-256 of the archived text
6e7ecd86d727f56b2976c17697c8fdabfbdc30e1d4b70dc54f00f0c2d8193446
Stable URL of this page
https://metraipolis.com/documento/dc_ca3d36a6845a

How to cite

DMAIC-Bench: From Self-Reflection to Self-Restraint for Agents in Multi-Stage Analytical Tasks — arXiv:2610.32305 — arXiv (en), https://arxiv.org/abs/2610.32305 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ca3d36a6845a (Metraipolis archive dc_ca3d36a6845a, sha256 6e7ecd86d727f56b2976c17697c8fdabfbdc30e1d4b70dc54f00f0c2d8193446)
BibTeX
@misc{dcca3d36a6845a,
  title = {DMAIC-Bench: From Self-Reflection to Self-Restraint for Agents in Multi-Stage Analytical Tasks — arXiv:2610.32305},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.32305},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ca3d36a6845a; sha256 6e7ecd86d727f56b2976c17697c8fdabfbdc30e1d4b70dc54f00f0c2d8193446},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.32305. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Multi-stage analytical tasks require LLM agents to use intermediate evidence to decide whether to continue, redirect or stop. Self-reflection, in which an agent critiques its output, is widely studied, yet a critique does not show whether evidence changed the next action. We call that capability self-restraint. Existing agent benchmarks largely measure task completion or output quality and leave it untested. DMAIC-Bench evaluates it with 97 seeded tasks across 15 industrial cases in four sectors, built around Lean Six Sigma's explicit statistical prerequisites. Paired counterfactual variants require opposite actions at the same workflow position, distinguishing warranted restraint from indiscriminate stopping. We assess execution traces with a private scorer and final claims through independent expert review. Across 13 models from six providers, execution-level routing accuracy averages 92% on matched GO decisions and 19% on NO-GO decisions, counting any forbidden downstream computation as a route error. Execution traces show agents reporting correct diagnostics before performing analyses that those results disallow. Separately, blinded expert review of the 11 audited agents estimates that reports act on invalid evidence at 47% of reached NO-GO decisions, with an interval from 31% to 61%. Explicit stopping permission helps some models without consistently restoring correct decisions at both GO and NO-GO forks. Sixteen life-sciences tasks in bulk ribonucleic acid sequencing (RNA-seq) and case-control genome-wide association studies (GWAS) show that the same framework can evaluate evidence-dependent decisions beyond industrial workflows. Project code and data are available at: this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.32305 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-26T19:45:33.304Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).