AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

18,070 stories · RSS feed

arXiv AI

Jul 21

What Do They See? Interpreting Complex Road Scenarios Through the Eyes of Vision-Language-Action Models for Safe and Trustworthy Autonomous Vehicle Learning

arXiv:2607. 16938v1 Announce Type: cross Abstract: End-to-end autonomous driving models are now able to navigate complex road scenarios, mapping raw sensor observations directly to observed paths for open-loop evaluation and often effective driving in closed-loop evaluation.

By Kalpana Panda, Wesley Maia, Vinti Agarwal, Ross Greer

agents robotics multimodal benchmarks safety

arXiv AI

Jul 21

Privacy Cost as Equity Input: A Group Fairness Criterion for Differentially Private Machine Learning

arXiv:2607. 16620v1 Announce Type: cross Abstract: Differential privacy (DP) is increasingly deployed to limit membership inference risk in machine-learning systems.

By Rakshit Naidu

nlp benchmarks safety

arXiv AI

Jul 21

Geometry-Enhanced Portion Estimation for Multimodal LLMs

arXiv:2607. 16514v1 Announce Type: cross Abstract: Image-based dietary assessment promises to replace costly, bias-prone manual recalls, but portion estimation remains a major blocker.

By Lin Liao, Peng Li

llms fine-tuning multimodal benchmarks safety

arXiv AI

Jul 21

Retrieval is Enough: Training-Free Interpretability with a Tool-Using Agent

arXiv:2607. 16448v1 Announce Type: cross Abstract: Interpretability methods for neural network activations span a wide cost spectrum, from cheap, training-free techniques (such as linear probes, PCA, SVD) to more expensive training-based ones (such as SAEs and activation oracles).

By Sriram Balasubramanian, Soheil Feizi

llms rag agents diffusion benchmarks safety

arXiv AI

Jul 21

PGN: Design and Implementation of a Vision-Language Navigation System Based on Pangu Multimodal Foundation Model

arXiv:2607. 17806v1 Announce Type: new Abstract: Vision-Language Navigation (VLN) requires an embodied agent to interpret a natural-language instruction and predict actions from temporally ordered visual observations.

By Li Xian, Mingxi Li, Yizheng Wang, Yiming Shen, Qi Chen, Zhuoling Xiao

llms rag agents robotics fine-tuning multimodal safety

arXiv AI

Jul 21

Dynamic Defense Profiling Enables Cognitive Jailbreak of Text-to-Image Models

arXiv:2607. 17779v1 Announce Type: new Abstract: Text-to-Image (T2I) generative models have achieved remarkable progress in synthesizing high-quality visual content, yet they remain vulnerable to adversarial misuse, particularly in generating Not-Safe-For-Work (NSFW) images.

By Dongdong Yang, Deyue Zhang, Zhao Liu, Zonghao Ying, Wenzhuo Xu, Jiankai Jin, Xiangzheng Zhang, Quanchen Zou

llms diffusion multimodal benchmarks safety

arXiv AI

Jul 21

The Curvature Shadow: An Apparent Failure of Maximum-Entropy Equilibrium Selection is a Removable Artifact

arXiv:2607. 17543v1 Announce Type: new Abstract: In two-player zero-sum games whose Nash equilibria form a convex set, regularized solvers such as Regularized Nash Dynamics (R-NaD) empirically select the maximum-entropy member: the information projection (I-projection) of a uniform reference onto the Nash set.

By Luis Leal

safety

arXiv AI

Jul 21

An Explicit World Model Based on Data-First Ontology: DaoQL Multimodal Storage Validation and Counterfactual Reasoning Evaluation

arXiv:2607. 17269v1 Announce Type: new Abstract: Large language models encode world models implicitly in neural weights, which exposes four structural risks in high-precision domains such as medicine and finance: hallucination, frozen knowledge, poor explainability, and poor modifiability.

By Zhanbo Li, Shifeng Wu, Xiangjin Meng, Wenjie Cai

llms agents multimodal benchmarks safety

arXiv AI

Jul 21

Is Your Model Thinking or Just Stagnating? PUMA: Diagnosing Reasoning Pathology via Phase-Momentum Alignment

arXiv:2607. 17188v1 Announce Type: new Abstract: Test-time scaling empowers Large Reasoning Models (LRMs) to tackle complex tasks via extensive Chain-of-Thought (CoT).

By Cheng Yan, Guangyang Ye, Wuyang Zhang, Fan Xu, Zhijun Fan, Xiang Xia, Yanyong Zhang

benchmarks safety

arXiv AI

Jul 21

Rater State Bias in RLHF Preference Data: An Audit Framework

arXiv:2607. 16195v1 Announce Type: new Abstract: We identify a structured confound in Reinforcement Learning from Human Feedback (RLHF).

By Elena Kopteva, Vitaliy Hlynianyi-Zhuk

reinforcement-learning safety

arXiv AI

Jul 21

Masked Diffusion Language Models are Strong and Steerable Text-Based World Models for Agentic RL

arXiv:2607. 16204v1 Announce Type: new Abstract: Recent growth in reinforcement learning (RL) has surfaced a need for diverse, specialized training environments.

By Darshan Deshpande

llms agents diffusion reinforcement-learning fine-tuning efficiency safety

arXiv Machine Learning

Jul 21

SpecXMaster Technical Report

arXiv:2603. 23101v3 Announce Type: replace Abstract: Intelligent spectroscopy serves as a pivotal element in AI-driven closed-loop scientific discovery, functioning as the critical bridge between matter structure and artificial intelligence.

By Yutang Ge, Yaning Cui, Hanzheng Li, Jun-Jie Wang, Fanjie Xu, Jinhan Dong, Yongqi Jin, Dongxu Cui, Peng Jin, Guojiang Zhao, Hengxing Cai, Tianci Yangfeng, Xueqing Chen, Hongshuai Wang, Rong Zhu, Linfeng Zhang, Xiaohong Ji, Zhifeng Gao

agents reinforcement-learning benchmarks safety

arXiv AI

Jul 21

"Not in My Backyard": LLMs Uncover Online and Offline Social Biases Against Homelessness

arXiv:2508. 13187v4 Announce Type: replace-cross Abstract: Homelessness is a persistent social challenge, impacting millions worldwide.

By Jonathan A. Karr Jr., Benjamin F. Herbst, Matthew L. Sisk, Xueyun Li, Ting Hua, Matthew Hauenstein, Georgina Curto, Nitesh V. Chawla

llms benchmarks safety

arXiv AI

Jul 21

Retain or Consolidate? Budget-Dependent Operator Selection for Language Agent Memory

arXiv:2607. 17545v1 Announce Type: new Abstract: Language agents depend on memory across interactions.

By Qingcan Kang, Mingyang Liu, Shixiong Kai, Kaichao Liang, Zhentao Tang, Yuqi Cui, Tao Zhong, Mingxuan Yuan

llms agents benchmarks safety

arXiv AI

Jul 21

Li-ViP3D++: Query-Gated Deformable Camera-LiDAR Fusion for End-to-End Perception and Trajectory Prediction

arXiv:2601. 20720v2 Announce Type: replace-cross Abstract: End-to-end perception and trajectory prediction from raw sensor data is one of the key capabilities for autonomous driving.

By Matej Halinkovic, Nina Masarykova, Alexey Vinel, Marek Galinski

agents robotics multimodal safety

arXiv AI

Jul 21

SONAR: Spectral-Contrastive Audio Residuals for Generalizable Deepfake Detection

arXiv:2511. 21325v2 Announce Type: replace-cross Abstract: Deepfake (DF) audio detectors still struggle to generalize to out of distribution inputs.

By Ido Nitzan Hidekel, Gal lifshitz, Khen Cohen, Dan Raviv

rag diffusion benchmarks safety

arXiv AI

Jul 21

Some Large Language Models Exhibit Consistent Risk Attitudes

arXiv:2607. 16197v1 Announce Type: new Abstract: As artificial intelligence systems are deployed in open-ended, high-stakes settings, a critical dimension remains unmeasured: how perceived risk is translated into action.

By Bowen Sun, Rui Min, Yuxi Wang, Brian Odegaard, Qi Wang, Jing Du

llms agents safety

arXiv Machine Learning

Jul 21

Brain-Aligned Multi-Stream Video Transformers with Sparse Self-Selection

arXiv:2607. 17625v1 Announce Type: cross Abstract: Modern video transformers typically ignore principles from primate vision and are rarely evaluated against neural data, limiting their biological interpretability.

By Amir Hosein Fadaei, Mahyar Maleki, Mohammad-Reza A. Dehaqani

llms rag safety

arXiv AI

Jul 21

PhysAgent: Reflective Agentic Physics Control for Physically Plausible Video Generation

arXiv:2607. 16355v1 Announce Type: cross Abstract: Recent advances in physics-grounded video generation leverage physics simulation as a physical prior to guide video synthesis toward physically plausible outcomes.

By Qirui Li, Jinkun Hao, Yibo Li, Ran Yi, Paul L. Rosin, Yu-Kun Lai

llms agents diffusion multimodal safety

arXiv AI

Jul 21

PlanFlip: Attacking Multi-Agent LLM Systems via Planning-Phase Prompt Injection

arXiv:2607. 16199v1 Announce Type: new Abstract: Multi-agent LLM systems increasingly rely on a Planner to decompose goals into sub-task sequences that downstream Executor and Critic agents execute and audit.

By Yuhang Wang

llms agents multimodal safety

Get the daily briefing