Metraipolis › archive › dc_954ebd5fc8a7
Evaluation of Multi-Turn Consistency in LLM Agents: Survival Analysis and Failure-Rationale Taxonomy — arXiv:2609.29508
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.29508
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-08T04:19:56.456Z
- SHA-256 of the archived text
357c1995759860f3cecf0aea971b07f32dfb8ca10aeab4506160af4f8b2f07de- Stable URL of this page
https://metraipolis.com/documento/dc_954ebd5fc8a7
How to cite
Evaluation of Multi-Turn Consistency in LLM Agents: Survival Analysis and Failure-Rationale Taxonomy — arXiv:2609.29508 — arXiv (en), https://arxiv.org/abs/2609.29508 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_954ebd5fc8a7 (Metraipolis archive dc_954ebd5fc8a7, sha256 357c1995759860f3cecf0aea971b07f32dfb8ca10aeab4506160af4f8b2f07de)
BibTeX
@misc{dc954ebd5fc8a7,
title = {Evaluation of Multi-Turn Consistency in LLM Agents: Survival Analysis and Failure-Rationale Taxonomy — arXiv:2609.29508},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.29508},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_954ebd5fc8a7; sha256 357c1995759860f3cecf0aea971b07f32dfb8ca10aeab4506160af4f8b2f07de},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.29508. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language model (LLM) agents may perform well on isolated tasks yet drift into inconsistency over extended interaction. We evaluate temporal consistency in a controlled 20-step multi-agent setting inspired by delayed-gratification studies. At each step, an agent chooses between continuing to delay a reward or claiming it immediately (terminating the episode). Across a full-factorial manipulation of social visibility (private vs public), persona stressors, and deliberation policy, we run 84,540 trajectories spanning 8 model families. Treating the first reward-claim as a time-to-event outcome, we estimate Kaplan-Meier survival curves and fit discrete-time hazard regression to quantify how experimental factors shift failure risk over time. Then, to analyze rationales and language patterns associated with failure, we build a seven-category taxonomy from 13,780 deliberation traces from agents who choose to terminate the episode, using an LLM-assisted labeling paired with human audit ($\kappa=0.83$). Rationale profiles change systematically with time and context: early failures are more impulse-driven, later failures more fatigue- and cost-benefit-framed, while public settings increase norm-oriented justifications. We also find a deliberation-inconsistency association: among failures, longer deliberation correlates with higher rates of intra-rationale contradiction (simultaneous pro-delay and pro-claim statements), challenging the assumption that more reasoning text implies greater consistency. Together, the survival and rationale analyses reveal distinct temporal reliability regimes and model-specific "failure fingerprints", offering an evaluation lens for diagnosing inconsistency in multi-turn agent behavior. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-25 in cs.MA). Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL); Machine Learning (cs.LG); Multiagent Systems (cs.MA). Pagina del lavoro: https://arxiv.org/abs/2609.29508 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-08T03:50:35.839Z dalla pagina https://arxiv.org/catchup/cs.MA/2026-09-25?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).