arXiv:2608. 13345v1 Announce Type: new Abstract: Artificial Intelligence (AI) safety systems combine character shaping (e.
arXiv:2410. 22526v2 Announce Type: replace Abstract: To effectively address potential harms from Artificial Intelligence (AI) systems, it is essential to identify and mitigate system-level hazards.
arXiv:2607. 16112v1 Announce Type: new Abstract: Frontier AI companies have published capability thresholds that differ substantially, making it difficult for third parties to verify whether a threshold has been crossed or to compare requirements across companies.
The paper introduces FUSE, a modular framework that evaluates large language models (LLMs) for dangerous capabilities across three orthogonal pipelines: Knowledge (K), Defense (D), and Harm (H). Using a chemical‑biological module, the authors assess 12 commercial LLMs, revealing divergent profiles among models and families, and showing that newer models increase knowledge while only partially improving defense. The framework’s reliability is supported by high cross‑judge consistency and low inter‑pipeline correlations.
ReSI is a recursive safety improvement framework that iteratively applies diverse red‑teaming methods to identify vulnerabilities in AI models, develops training recipes, and selects the update with the greatest safety gain while retaining capability. Across four dense and mixture‑of‑experts models, ReSI matches or surpasses frontier models on both in‑distribution and out‑of‑distribution safety benchmarks, and outperforms alignment baselines on nearly all safety evaluations. The framework notably reduces the mean X‑Teaming attack success rate from 86.01% to 31.45%, outperforming GPT‑5.6‑Luna’s 56.69% and demonstrating stronger resilience to unseen attacks.
arXiv:2609.38411v1 Announce Type: new Abstract: Leading AI developers have reported agents acting beyond their approved limits, which a United Nations panel described as an early warning of loss of h...
The paper proves that using a permissive safety filter in reinforcement learning does not compromise asymptotic performance. By formalizing safety through a safety‑critical Markov decision process and a filtered MDP, the authors show that optimal policies in the filtered MDP achieve the same return as the best safe policy in the original setting. Experiments on Safety Gymnasium confirm zero violations during training and performance that matches or exceeds unfiltered baselines.
The paper proposes a structured method for assessing the representativeness of Operational Design Domains (ODDs) in AI/ML-based aviation systems, focusing on safety assurance. It outlines a process flow from ODD definition to quantitative evaluation, recommending Kullback–Leibler divergence and Cramér’s V over chi‑squared tests for large datasets. The approach is illustrated with AI-based collision avoidance simulations, demonstrating how statistical distribution comparisons can support safety‑by‑design engineering aligned with EASA guidance.
The paper investigates safe reinforcement learning by focusing on the episodic-cost tail rather than just the mean cost. It introduces the use of γαρα_{0.1} (the average cost of the worst 10% of episodes) to classify policies as tail-safe. The authors evaluate standard algorithms on Safety-Gymnasium tasks, identify tail-unsafe policies, and explore constraint families to control tail violations while maintaining return.
FAITH is a model‑free reinforcement learning framework that separates safety from task performance by using a learned safety filter. The filter approximates the optimal state‑action safety value with a feedforward network, allowing the task policy to optimize return through filtered dynamics without a competing safety term. Experiments on a double integrator, Safety Gym, a 29‑DoF humanoid, and a real Unitree G1 show that FAITH achieves high returns while maintaining near‑perfect safety and handling infeasible starts gracefully.
arXiv:2606. 08919v1 Announce Type: new Abstract: As LLM agents begin to take real, irreversible actions (shell commands, file edits, deploys), the standard safety pattern is a human-in-the-loop approval gate: risky actions pause and wait for a person.
The paper introduces the concept of Evolutionary Safety for recursive self-improving AI, focusing on how safety properties evolve as an AI system and its successors change. It identifies key risks such as intent drift, error accumulation, and safety-property erosion, and presents a taxonomy covering agent state, model state, evaluation, environment, and update mechanisms. The authors propose methods for discovering and evaluating evolutionary risks, and outline governance principles for modification, selection, authorization, provenance, and recovery, while highlighting open problems for maintaining safety in persistent, adaptive, and recursively self-improving systems.
arXiv:2607. 12792v1 Announce Type: cross Abstract: Jailbreak-robustness research typically evaluates safety through generated responses using an LLM-as-judge approach.