AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

10,089 stories · RSS feed

arXiv AI
Jul 7

Open Problems in AI Incident Governance

arXiv:2607. 05163v1 Announce Type: cross Abstract: AI systems may produce failures after deployment that pre-deployment safety assessments do not anticipate.

By Harleen Kaur Sidhu, Rebecca Scholefield, Nour Annan, Kevin Hernandez, Isabel Nieh Hou, Abdulrahman Alshaikhi, Ze Shen Chin, Rokas Gipi\v{s}kis
arXiv Machine Learning
Jul 7

Non-Asymptotic Error Bounds for SMC with Biased Proposals: Application to Conditional Diffusion Sampling

arXiv:2607. 04780v1 Announce Type: cross Abstract: Sequential Monte Carlo (SMC) methods are a natural tool for post-hoc conditioning of pretrained generative models, but in many applications the mutation kernels used by the particle system are biased approximations of an ideal Feynman--Kac flow.

By Stanislas Strasman (SU, LPSM), Gabriel Victorino Cardoso (LPSM), Sylvain Le Corff (LPSM), Vincent Lemaire (LPSM), Antonio Ocello
arXiv AI
Jul 7

Decentralised Federated Learning over Temporal Networks: The Role of Heterogeneities

arXiv:2607. 03171v1 Announce Type: cross Abstract: Decentralised federated learning, based on peer-to-peer communication, is increasingly proposed for on-device training of machine learning models, promising a privacy-preserving, communication-efficient training process with no risk of single-point failure.

By Arash Badie-Modiri, Chiara Boldrini, Lorenzo Valerio, J\'anos Kert\'esz, M\'arton Karsai
arXiv AI
Jul 7

MentalThink: Shaping Thoughts in Mental SVG World

arXiv:2607. 03530v1 Announce Type: new Abstract: We introduce MentalThink, a visual-symbolic reasoning paradigm that equips Multimodal LLMs (MLLMs) with an executable mechanism for "mental" visualization.

By Kangheng Lin, Jisheng Yin, Dingming Li, En Yu, Yana Wei, Han Zhou, Liang Zhao, Hongyu Zhou, Hongbo Peng, Jianjian Sun, Zheng Ge, Xiangyu Zhang, Daxin Jiang, Jingyu Wang
arXiv AI
Jul 7

dOPSD: On-Policy Self-Distillation for Diffusion Language Models

arXiv:2607. 04428v1 Announce Type: cross Abstract: Diffusion large language models (dLLMs) generate text by iteratively denoising a masked sequence, offering a parallel alternative to autoregressive models, but eliciting strong reasoning through post-training remains difficult: supervised fine-tuning is off-policy and suffers from exposure bias, while reinforcement learning gives only sparse, sequence-level rewards and is hard to apply without tractable sequence likelihoods.

By Phuong Tuan Dat, Qi Li, Xinchao Wang
arXiv AI
Jul 7

SoftVTBench: A Safety-Aware Visuo-Tactile Benchmark for Physically Constrained Robotic Manipulation of Deformable Objects

arXiv:2607. 04234v1 Announce Type: cross Abstract: Deformable object manipulation poses challenges beyond task completion: successful execution must also maintain safe physical interaction, holding the object stably without slip or drop while avoiding excessive deformation.

By Bowen Jing, Mingxin Wang, Ruiyang Hao, Chenchen Ge, Hanwen Shen, Junjie He, Yang Cui, Yiming Hou, Weitao Zhou, Jiawei Wang, Minglei Li, Dandan Zhang, Ding Zhao, Houde Liu, Xiaofan Li, Si Liu, Ping Luo, Haibao Yu