FLY-EVAL++ is an evidence-driven evaluation protocol designed for safety-constrained flight prediction with large language models. It combines deterministic verification of protocol compliance, physical feasibility, and safety constraints with rubric-guided aggregation into interpretable multi-dimensional scores. Applied to Flight Trajectory and Attitude Prediction, the protocol revealed that safety compliance is the most discriminative dimension among 66 LLMs, with models showing up to 28-point differences in safety scores and recurrent failures such as safety violations under physically plausible predictions and instability in multi-step rollouts.
By Yalun Wu, Junfeng Fang, Jiawei Wang, Haotian Liu, Qijun Yang, Minghan Yang, Hongcheng Guo, Zhoujun Li, Boyang Wang
FLY-EVAL++ is an evidence-driven evaluation protocol designed for safety-constrained flight prediction with large language models. It combines deterministic verification of protocol compliance, physical feasibility, and safety constraints, then aggregates results into interpretable multi-dimensional scores. Applied to Flight Trajectory and Attitude Prediction, the protocol reveals that safety compliance is the most discriminative metric, with models of similar predictive accuracy differing by over 28 points in safety score and exhibiting recurrent safety violations and instability in multi-step rollouts.
Large language model (LLM) agents may assist flight crews with complex decisions and task execution, but existing aviation evaluations centered on static knowledge do not support systematic testing of...
arXiv:2608. 16349v1 Announce Type: new Abstract: Large language model (LLM) agents may assist flight crews with complex decisions and task execution, but existing aviation evaluations centered on static knowledge do not support systematic testing of procedural execution and safety compliance in interactive environments.
By Yuchen Yuan, Zhenghuang Wu, Yuangan Li, Liang Ma, Ke Li
arXiv:2608. 20053v1 Announce Type: new Abstract: The integration of Artificial Intelligence (AI) in safety-critical aviation systems presents significant challenges for certification and deployment.
By Johann Maximilian Christensen, Thomas Stefani, Elena Hoemann, Frank K\"oster, Sven Hallerbach
arXiv:2608.24621v2 Announce Type: replace
Abstract: Can language models be trusted in safety- critical operations? In such settings, strong per- formance on semantic metrics does not guaran- tee oper...
By Yujing Chang, Thinh Pham, Van-Phat Thai, Chunyao Ma, Yash Guleria, Pham Nhut Huy, Sameer Alam
The study surveyed maritime professionals on their attitudes toward AI‑supported decision assistants in collision‑avoidance scenarios. Results show a generally positive disposition toward maritime technology, stable trust across scenarios, and nuanced, scenario‑sensitive ratings of explanation quality. Open‑ended feedback highlighted the importance of decision‑support, situational awareness, and confidence‑building, while raising concerns about AI reliability, over‑reliance, and loss of expertise.
By Doreen Jirak, Armeen Saroukanoff, Dirk van Rooy
arXiv:2509.15959v2 Announce Type: replace-cross
Abstract: Autonomous navigation in maritime domains is accelerating alongside advances in artificial intelligence, sensing, and connectivity. Opaque de...
By Zhuoyue Zhang, Haitong Xu, Carlos Guedes Soares
arXiv:2607. 01829v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly proposed for aviation business operations, from documentation and training generation to customer facing assistants.
By Alex Brooker, Tim Hughes
arXiv:2608. 09857v1 Announce Type: cross Abstract: Advances in advanced artificial intelligence tools have sparked research in robot autonomy, but the development of such systems has largely focused on execution rather than verifying the feasibility actions planning models propose.
By Rohan Bhagra, Mahantesh Halapannavar, Uddhav Bhattarai
arXiv:2608. 04697v1 Announce Type: new Abstract: Operational hazard analysis of aviation system operations must consider interactions among weather, ATC actions, airspace constraints, aircraft operations, and human factors - distinct from the functional hazard assessment applied at the aircraft-system level.
By Cristian Mascia, Roberto Pietrantuono, Daniel Rodriguez, Stefano Russo
arXiv:2604. 02478v2 Announce Type: replace Abstract: Deep learning models excel at detecting anomaly patterns in normal data.
By Jiyong Kwon, Ujin Jeon, Sooji Lee, Guang Lin