AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

9,515 stories · RSS feed

arXiv AI

Jul 21

SyriSign: A Parallel Corpus for Arabic Text to Syrian Arabic Sign Language Translation

arXiv:2603. 29219v2 Announce Type: replace-cross Abstract: Sign language is the primary approach of communication for the Deaf and Hard-of-Hearing (DHH) community.

By Mohammad Amer Khalil, Raghad Nahas, Ahmad Nassar, Khloud Al Jallad

llms rag benchmarks safety

arXiv AI

Jul 21

DSBench: A Comprehensive Benchmark for Evaluating External and In-Cabin Risks

arXiv:2511. 14592v3 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) show great promise for autonomous driving, but their suitability for safety-critical scenarios is largely unexplored, raising safety concerns.

By Xianhui Meng, Yuchen Zhang, Zhijian Huang, Zheng Lu, Ziling Ji, Yandan Lin, Yaoyao Yin, Hongyuan Zhang, Wei Zhou, Guangfeng Jiang, Li Zhang, Long Chen, Hangjun Ye, Jun Liu, Xiaoshuai Hao

llms agents robotics fine-tuning multimodal benchmarks safety

arXiv AI

Jul 21

Symmetric Behavior Regularized Policy Optimization

arXiv:2508. 04225v4 Announce Type: replace-cross Abstract: Behavior Regularized Policy Optimization (BRPO) leverages asymmetric divergence regularization to mitigate distribution shift in offline reinforcement learning.

By Lingwei Zhu, Haseeb Shah, Zheng Chen, Martha White

reinforcement-learning benchmarks safety

arXiv AI

Jul 21

SCA: Segment-Wise CoT Compression with Answer Alignment

arXiv:2603. 07598v2 Announce Type: replace Abstract: Chain-of-thought (CoT) reasoning improves problem solving, but long think traces increase inference cost.

By Ye Tian, Hongyu Lin

benchmarks safety

arXiv AI

Jul 21

Enhancing LLMs' Clinical Reasoning with Real-World Data from a Nationwide Sepsis Registry

arXiv:2505. 02722v2 Announce Type: replace Abstract: Although large language models (LLMs) have demonstrated impressive reasoning capabilities across general domains, their effectiveness in real-world clinical practice remains limited.

By Junu Kim, Chaeeun Shim, Sungjin Park, Su Yeon Lee, Gee Young Suh, Chae-Man Lim, Seong Jin Choi, Song Mi Moon, Kyoung-Ho Song, Eu Suk Kim, Hong Bin Kim, Sejoong Kim, Chami Im, Dong-Wan Kang, Yong Soo Kim, Hee-Joon Bae, Sung Yoon Lim, Han-Gil Jeong, Edward Choi

llms reinforcement-learning fine-tuning safety

arXiv AI

Jul 21

BUSTR: Descriptor-Aware Vision-Language Learning for Breast Ultrasound Report Generation

arXiv:2511. 20956v2 Announce Type: replace-cross Abstract: Breast ultrasound (BUS) reporting relies on clinically meaningful lesion descriptors, including BI-RADS category, lesion shape, margin, echogenicity, posterior features, pathology, and histology.

By Rawa Mohammed, Mina Attin, Laxmi Gewali, Bryar Shareef

llms multimodal safety

arXiv Machine Learning

Jul 21

AutoEncoder-Compressed Parallel Split Learning for Pre-trained Model Fine-Tuning

arXiv:2607. 17913v1 Announce Type: cross Abstract: Distributed Fine-Tuning (DFT) of large-scale Foundation Models (FMs) on resource-constrained edge devices is limited by local compute constraints and communication overhead.

By Bas Meuwissen, Vasileios Tsouvalas, Nirvana Meratnia

fine-tuning efficiency safety

arXiv Machine Learning

Jul 21

Bigger Is Safer: Provable Robustness in In-Context Learning Scales with Capacity

arXiv:2602. 17743v2 Announce Type: replace Abstract: In-context learning (ICL) allows large language models to adapt to new tasks from a few examples without updating their parameters.

By Di Zhang, Ningxu Zhang, Zimeng Liu

llms safety

arXiv Machine Learning

Jul 21

Sockpuppetting: Jailbreaking LLMs by Combining Prefilling with Optimization

arXiv:2601. 13359v3 Announce Type: replace-cross Abstract: Prefill attacks are an effective and low-cost jailbreaking method, as they directly insert an acceptance sequence (e.

By Asen Dotsinski, Panagiotis Eustratiadis

llms multimodal benchmarks safety

arXiv AI

Jul 21

A Survey on the Verification of Reinforcement Learning Policies

arXiv:2607. 16210v1 Announce Type: new Abstract: Reinforcement learning (RL) is increasingly applied in complex, safety-critical domains, yet the lack of rigorous behavioral guarantees for neural network-based policies remains a major barrier to deployment.

By Luca Marzari, Ezio Bartocci, Enrico Marchesini

reinforcement-learning safety

arXiv AI

Jul 21

Some Large Language Models Exhibit Consistent Risk Attitudes

arXiv:2607. 16197v1 Announce Type: new Abstract: As artificial intelligence systems are deployed in open-ended, high-stakes settings, a critical dimension remains unmeasured: how perceived risk is translated into action.

By Bowen Sun, Rui Min, Yuxi Wang, Brian Odegaard, Qi Wang, Jing Du

llms agents safety

arXiv AI

Jul 21

Reward-Driven LLM Agent Workflows: Synthesizing POMDP Routing and Self-Correction for Autonomous Decision-Making

arXiv:2607. 17038v1 Announce Type: new Abstract: This paper addresses key technical challenges in current large language model (LLM) agent applications, including long-horizon planning, sparse reward attribution, and dynamic environmental interaction, by designing and optimizing an intelligent agent workflow.

By Amez Amanj Ali, Kuo-Kun Tseng

llms agents reinforcement-learning robotics multimodal benchmarks safety

arXiv AI

Jul 21

Is Your Model Thinking or Just Stagnating? PUMA: Diagnosing Reasoning Pathology via Phase-Momentum Alignment

arXiv:2607. 17188v1 Announce Type: new Abstract: Test-time scaling empowers Large Reasoning Models (LRMs) to tackle complex tasks via extensive Chain-of-Thought (CoT).

By Cheng Yan, Guangyang Ye, Wuyang Zhang, Fan Xu, Zhijun Fan, Xiang Xia, Yanyong Zhang

benchmarks safety

arXiv AI

Jul 21

An Explicit World Model Based on Data-First Ontology: DaoQL Multimodal Storage Validation and Counterfactual Reasoning Evaluation

arXiv:2607. 17269v1 Announce Type: new Abstract: Large language models encode world models implicitly in neural weights, which exposes four structural risks in high-precision domains such as medicine and finance: hallucination, frozen knowledge, poor explainability, and poor modifiability.

By Zhanbo Li, Shifeng Wu, Xiangjin Meng, Wenjie Cai

llms agents multimodal benchmarks safety

arXiv AI

Jul 21

Dynamic Defense Profiling Enables Cognitive Jailbreak of Text-to-Image Models

arXiv:2607. 17779v1 Announce Type: new Abstract: Text-to-Image (T2I) generative models have achieved remarkable progress in synthesizing high-quality visual content, yet they remain vulnerable to adversarial misuse, particularly in generating Not-Safe-For-Work (NSFW) images.

By Dongdong Yang, Deyue Zhang, Zhao Liu, Zonghao Ying, Wenzhuo Xu, Jiankai Jin, Xiangzheng Zhang, Quanchen Zou

llms diffusion multimodal benchmarks safety

arXiv AI

Jul 21

PGN: Design and Implementation of a Vision-Language Navigation System Based on Pangu Multimodal Foundation Model

arXiv:2607. 17806v1 Announce Type: new Abstract: Vision-Language Navigation (VLN) requires an embodied agent to interpret a natural-language instruction and predict actions from temporally ordered visual observations.

By Li Xian, Mingxi Li, Yizheng Wang, Yiming Shen, Qi Chen, Zhuoling Xiao

llms rag agents robotics fine-tuning multimodal safety

arXiv AI

Jul 21

RobustMAD: Evaluating Real-World Robustness of Multimodal Small Language Models for Deployable Anomaly Detection Assistants

arXiv:2607. 16243v1 Announce Type: cross Abstract: Multimodal industrial anomaly inspection assistants are a critical component of next-generation smart factories, enabling interactive vision-language-based querying.

By Anushiya Arunan, Xin Li, Yan Qin, U-Xuan Tan, Nhu Khue Vuong, Xiaoli Li, Chau Yuen

llms multimodal benchmarks safety

arXiv AI

Jul 21

Autonomous mechanistic discovery of colorectal cancer vulnerabilities via multi-scale AI swarms

arXiv:2607. 16262v1 Announce Type: cross Abstract: The acceleration of automated scientific discovery has been fundamentally bottlenecked by the epistemic gap between the semantic reasoning of large language models (LLMs) and the deterministic physics of mammalian biology.

By Christopher Baker, Tianyu Ren, Karen Rafferty, Hui Wang, Simon McDade

llms agents safety

arXiv AI

Jul 21

Privacy Cost as Equity Input: A Group Fairness Criterion for Differentially Private Machine Learning

arXiv:2607. 16620v1 Announce Type: cross Abstract: Differential privacy (DP) is increasingly deployed to limit membership inference risk in machine-learning systems.

By Rakshit Naidu

nlp benchmarks safety

arXiv AI

Jul 21

What Do They See? Interpreting Complex Road Scenarios Through the Eyes of Vision-Language-Action Models for Safe and Trustworthy Autonomous Vehicle Learning

arXiv:2607. 16938v1 Announce Type: cross Abstract: End-to-end autonomous driving models are now able to navigate complex road scenarios, mapping raw sensor observations directly to observed paths for open-loop evaluation and often effective driving in closed-loop evaluation.

By Kalpana Panda, Wesley Maia, Vinti Agarwal, Ross Greer

agents robotics multimodal benchmarks safety

Get the daily briefing