Metraipolis › archive › dc_ced2b463e10a
FLY-EVAL++: An Evidence-Driven Evaluation Protocol for Safety-Constrained Flight Prediction with Large Language Models — arXiv:2609.04021
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.04021
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-17T01:11:30.603Z
- SHA-256 of the archived text
92d58a27d4935b4411527a0c1ad407367e1f3d81fb096e0a29b4209d6ae26096- Stable URL of this page
https://metraipolis.com/documento/dc_ced2b463e10a
How to cite
FLY-EVAL++: An Evidence-Driven Evaluation Protocol for Safety-Constrained Flight Prediction with Large Language Models — arXiv:2609.04021 — arXiv (en), https://arxiv.org/abs/2609.04021 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ced2b463e10a (Metraipolis archive dc_ced2b463e10a, sha256 92d58a27d4935b4411527a0c1ad407367e1f3d81fb096e0a29b4209d6ae26096)
BibTeX
@misc{dcced2b463e10a,
title = {FLY-EVAL++: An Evidence-Driven Evaluation Protocol for Safety-Constrained Flight Prediction with Large Language Models — arXiv:2609.04021},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.04021},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ced2b463e10a; sha256 92d58a27d4935b4411527a0c1ad407367e1f3d81fb096e0a29b4209d6ae26096},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.04021. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Evaluating large language models (LLMs) in safety-critical, physics-governed environments requires more than accuracy-based metrics, because predictions that are numerically close to the ground truth can still violate operational constraints, combine fields in physically inconsistent ways, or fail to produce usable structured outputs. Existing evaluation protocols do not measure these failure modes reliably. We propose FLY-EVAL++, an evidence-driven evaluation protocol that combines deterministic verification of protocol compliance, physical feasibility, and safety constraints with fixed rubric-guided aggregation into interpretable multi-dimensional scores. We instantiate FLY-EVAL++ for Flight Trajectory and Attitude Prediction (FTAP) by extending the PilotBench setting with history-conditioned and multi-step prediction tasks. Across 66 LLMs, safety compliance is the most discriminative dimension of model behavior: models with comparable predictive performance differ by more than 28 points in safety score, and we observe recurrent failures including safety violations under physically plausible predictions and instability in multi-step rollouts. These results show that evaluation in safety-critical domains should measure constraint satisfaction and structured validity explicitly rather than rely on accuracy-centric reporting alone. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-04 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.04021 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-17T00:28:02.902Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-04?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).