Metraipolis › archive › dc_dbba8ce62bd5
GraphACM: Graph-based Action Consistency Measurement for Uncertainty Quantification in LLM Agents — arXiv:2610.22031
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.22031
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T09:04:11.214Z
- SHA-256 of the archived text
90c267b2dc9d078beda0adcfba1e716bee34220fe44f2c33a87772baf4276ab1- Stable URL of this page
https://metraipolis.com/documento/dc_dbba8ce62bd5
How to cite
GraphACM: Graph-based Action Consistency Measurement for Uncertainty Quantification in LLM Agents — arXiv:2610.22031 — arXiv (en), https://arxiv.org/abs/2610.22031 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_dbba8ce62bd5 (Metraipolis archive dc_dbba8ce62bd5, sha256 90c267b2dc9d078beda0adcfba1e716bee34220fe44f2c33a87772baf4276ab1)
BibTeX
@misc{dcdbba8ce62bd5,
title = {GraphACM: Graph-based Action Consistency Measurement for Uncertainty Quantification in LLM Agents — arXiv:2610.22031},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.22031},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_dbba8ce62bd5; sha256 90c267b2dc9d078beda0adcfba1e716bee34220fe44f2c33a87772baf4276ab1},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.22031. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Behavioral consistency across repeated executions provides an informative signal for assessing the reliability of large language model (LLM) agents. However, existing approaches typically summarize consistency with a single task-level score, without identifying where behavioral differences occur or whether actions are compared under equivalent decision contexts. We propose a graph-based framework for fine-grained behavioral consistency measurement and uncertainty quantification. Given multiple rollouts of the same task, we merge equivalent states into a directed, action-labeled graph, enabling action comparisons at shared decision states. We introduce node-level and global measures of action consistency, using the global consistency score as a proxy for uncertainty to predict task success or failure. Experiments across three LLMs on ALFWorld, AppWorld, and WebShop demonstrate that our measure is positively associated with task success and achieves an average AUROC of 0.811 for task-success prediction, outperforming the strongest behavioral-consistency baseline (0.729) and verbalized confidence (0.769). These results highlight the effectiveness of state-conditioned behavioral consistency and pinpoint where agent actions agree. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.22031 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T08:44:17.803Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).