Metraipolis › archive › dc_fcd3cd2ed5d6

Signals: Continuous Diagnostics of AI Agents — arXiv:2611.04507

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.04507
License (as archived): CC0-1.0
Archived: 2026-11-04T12:28:27.724Z
SHA-256 of the archived text: 05bac0fca69f36fe309c326e9797a5a8f8d8e31dd72495979754e1b79e5f412c
Stable URL of this page: https://metraipolis.com/documento/dc_fcd3cd2ed5d6

How to cite

Signals: Continuous Diagnostics of AI Agents — arXiv:2611.04507 — arXiv (en), https://arxiv.org/abs/2611.04507 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_fcd3cd2ed5d6 (Metraipolis archive dc_fcd3cd2ed5d6, sha256 05bac0fca69f36fe309c326e9797a5a8f8d8e31dd72495979754e1b79e5f412c)

BibTeX

@misc{dcfcd3cd2ed5d6,
  title = {Signals: Continuous Diagnostics of AI Agents — arXiv:2611.04507},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.04507},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_fcd3cd2ed5d6; sha256 05bac0fca69f36fe309c326e9797a5a8f8d8e31dd72495979754e1b79e5f412c},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.04507. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

AI agents can make incorrect decisions or waste resources even when their tools and infrastructure work normally. Their nondeterministic behavior makes these problems difficult to reproduce and diagnose. We developed Signals to identify such problems from agentic traces and suggest concrete remediations. We built the method around a taxonomy of 45 problem types across eight families, with definitions and evidence requirements for each type. The method distinguishes realized failures, risk exposures, and inefficiencies, groups related findings into engineering issues called Signals, and assesses their severity from the supporting evidence. Each Signal includes actions a developer can take and checks for evaluating the change. Further executions can add evidence to an existing Signal or reveal a new problem. We evaluated the method on public agent executions and achieved 91.43% F1 when matching Signals to reference issues, about 18 percentage points higher than a one-pass LLM control that used the same model and taxonomy.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.04507 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-04T12:26:47.162Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).