AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

10,825 stories · RSS feed

arXiv AI
Jun 6

Reward-Decomposed Reinforcement Learning for Immersive Video Role-Playing

arXiv:2605. 04733v2 Announce Type: replace Abstract: Text-based role-playing models can imitate character styles, but often fail to capture scene atmosphere and evolving tension, which are crucial for immersive applications such as VR games and interactive narratives.

By Miao Wang, Yuling Shi, Yijiang Li, Yeheng Chen, Xiaodong Gu, Bin Li, Bo Gao, Jun Wang, Zengxin Han, Jingtong Wu, Yaduan Ruan
arXiv AI
Jun 6

Retrieval-Augmented Generation Must Move Beyond Factual Grounding to Represent Diverse Opinions

arXiv:2604. 12138v2 Announce Type: replace Abstract: This position paper argues that Retrieval-Augmented Generation systems exhibit a systematic factual bias-optimizing for epistemic uncertainty reduction while ignoring the aleatoric uncertainty inherent in opinion-rich content - and that this misalignment demands a paradigm shift in retrieval system design.

By Aditya Agrawal, Alwarappan Nakkiran, Darshan Fofadiya, Alex Karlsson, Harsha Aduri
arXiv AI
Jun 6

Output Type Before Quality: A Standards-Derived XAI Admissibility Rubric for Autonomous-Driving Safety

arXiv:2606. 05461v1 Announce Type: new Abstract: Safety standards for ML-based autonomous driving specify the kind of evidence an assurance case must contain (directed cause-and-effect chains, quantified interventional effects, named root-cause variables), yet the XAI literature is organised by output type and technique family (saliency maps, feature attribution, counterfactuals, causal graphs, language traces).

By Abhinaw Priyadershi, Mandar Pitale, Jelena Frtunikj, Maria Spence
arXiv AI
Jun 6

Risk Assessment of Autonomous Driving: Integrating Technical Failures, Ethical Dilemmas, and Policy Frameworks

arXiv:2606. 06396v1 Announce Type: new Abstract: Autonomous driving technology has the potential to reduce the large number of road traffic accidents caused by human error each year, but it also brings new types of risks that need to be evaluated from the aspects of technology, ethics and regulations.

By Boyi Chen, Shengqin Chu, Zicheng Wang, Brian Baetz, Zhen Gao
arXiv AI
Jun 6

Willing but Unable: Separating Refusal from Capability in Code LLMs via Abliteration

arXiv:2606. 05396v1 Announce Type: cross Abstract: Producing a labeled vulnerable code at scale is a recurring obstacle for learning-based vulnerability detection: mined corpora carry substantial label noise, and existing LLM-based augmentation propagates these inaccuracies because it transforms vulnerable seeds rather than synthesising vulnerabilities from a specification.

By Cristina Carleo, Pietro Liguori, Naghmeh Ivaki, Domenico Cotroneo
arXiv Machine Learning
Jun 5

Can LLMs Write Correct TLA+ Specifications? Evaluating Natural-Language-to-TLA+ Generation

arXiv:2606. 05792v1 Announce Type: cross Abstract: TLA+ has supported industrial verification at companies such as Amazon and Microsoft, yet writing correct TLA+ specifications from natural language still requires time and expertise, which limits adoption.

By Arslan Bisharat, Brian Ortiz, Eric Spencer, Khushboo Bhadauria, TaiNing Wang, George K. Thiruvathukal, Konstantin Laufer, Mohammed Abuhamad
arXiv Machine Learning
Jun 5

Bridging Domain Expertise and Generalization for Performance Estimation

arXiv:2606. 06335v1 Announce Type: new Abstract: Performance estimation under distribution shift aims to predict how a model behaves on an unlabeled test set whose distribution differs from the training data, a scenario that requires reliable indicators that can faithfully reflect model behavior without ground-truth labels.

By Shuxuan Li, Zhilin Zhao, Quyu Kong, Wei-Shi Zheng
arXiv Machine Learning
Jun 5

Merging model-based control with multi-agent reinforcement learning for multi-agent cooperative teaming strategies

arXiv:2606. 06011v1 Announce Type: cross Abstract: In this work, we propose a framework that combines multi-agent reinforcement learning (MARL) with model-based control to achieve safe, dynamically feasible actions in cooperative multi-agent tasks.

By Christian Llanes, Spencer W. Jensen, Samuel Coogan
arXiv Machine Learning
Jun 5

Quantifying Sensitivity for Tree Ensembles: A symbolic and compositional approach

arXiv:2605. 13830v2 Announce Type: replace-cross Abstract: Decision tree ensembles (DTE) are a popular model for a wide range of AI classification tasks, used in multiple safety critical domains, and hence verifying properties on these models has been an active topic of study over the last decade.

By Ajinkya Naik, Chaitanya Garg, S. Akshay, Ashutosh Gupta, Kuldeep S. Meel