Metraipolis › archive › dc_58c574d17cbd

Too Sure to Be Safe: Model Calibration for Reliable Log Anomaly Detection — arXiv:2608.17965

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.17965
License (as archived)
CC0-1.0
Archived
2026-10-18T14:33:48.612Z
SHA-256 of the archived text
13f3936ac5ecc79b64bec4b628bcf37ad2f3ce5c59747dc22f85f3c8300a4447
Stable URL of this page
https://metraipolis.com/documento/dc_58c574d17cbd

How to cite

Too Sure to Be Safe: Model Calibration for Reliable Log Anomaly Detection — arXiv:2608.17965 — arXiv (en), https://arxiv.org/abs/2608.17965 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_58c574d17cbd (Metraipolis archive dc_58c574d17cbd, sha256 13f3936ac5ecc79b64bec4b628bcf37ad2f3ce5c59747dc22f85f3c8300a4447)
BibTeX
@misc{dc58c574d17cbd,
  title = {Too Sure to Be Safe: Model Calibration for Reliable Log Anomaly Detection — arXiv:2608.17965},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.17965},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_58c574d17cbd; sha256 13f3936ac5ecc79b64bec4b628bcf37ad2f3ce5c59747dc22f85f3c8300a4447},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.17965. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Online log anomaly detection is critical for maintaining the reliability of large-scale computing systems. Although recent language model-based log anomaly detectors achieve strong detection performance, their confidence estimates remain poorly calibrated. We show that these detectors frequently assign excessive confidence to incorrect predictions, particularly for anomalous logs under severe class imbalance. Moreover, confidence on erroneous predictions remains persistently high even when conventional calibration metrics indicate good calibration, creating a critical reliability gap for operational monitoring systems. To address this issue, we propose Log Reconstruction and Distance (LoRD), a lightweight post-hoc calibration framework for reliable log anomaly detection. LoRD learns prediction-route-specific reliability models from latent representations of correctly classified validation samples and estimates prediction reliability through route-wise reconstruction distances. Based on the estimated reliability, LoRD selectively recalibrates high-risk predictions to suppress overconfident errors while preserving reliable predictions. Extensive experiments on four large-scale log benchmark datasets and multiple language model-based detectors demonstrate that LoRD consistently improves confidence reliability and substantially reduces overconfident anomaly-related errors without sacrificing anomaly detection performance.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-19 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Software Engineering (cs.SE).
Pagina del lavoro: https://arxiv.org/abs/2608.17965 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T14:27:54.475Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-19?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).