AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,788 stories · RSS feed

arXiv Statistics ML

Oct 27

HIFI: A Highly Efficient Framework for Statistical Inference with Foundation Models

arXiv:2610.34550v1 Announce Type: new Abstract: Foundation models provide powerful representations for unstructured data, such as text and images, and parameter-efficient fine-tuning (PEFT) makes the...

By Po-Wei Chen, Shan Yu, Chunlin Li

fine-tuning safety

arXiv Statistics ML

Oct 27

Minimax Regret for Sobolev Kernel-Based Reinforcement Learning

arXiv:2610.34045v1 Announce Type: cross Abstract: We study the minimax regret rate for episodic kernel-based reinforcement learning, where the expected reward and transition density belong to a Sobol...

By Haichen Hu, David Simchi-Levi

reinforcement-learning safety

arXiv Statistics ML

Oct 27

Risk-Aware Output Control Should be the Focus of Machine Unlearning

arXiv:2610.35052v1 Announce Type: cross Abstract: Well-documented failure modes of machine learning (ML) systems, including bias, toxicity, hallucination, and leakage of sensitive information, has dr...

By Yahya Alkhatib, Yuyang Xiao, Wee Peng Tay

safety

arXiv Statistics ML

Oct 27

Adaptive Estimation and Inference in Conditional Moment Models via the Discrepancy Principle

arXiv:2603.01337v2 Announce Type: replace Abstract: We study adaptive estimation and inference in linear inverse problems defined by conditional moment restrictions. These problems are typically ill-...

By Jiyuan Tan, Vasilis Syrgkanis

safety

arXiv AI

Oct 27

Artificial Intelligence in Mechanical Engineering: Foundations, Opportunities, Challenges, and Responsibilities

arXiv:2610.33105v1 Announce Type: new Abstract: Artificial intelligence (AI) is reshaping mechanical engineering (ME), from modeling, simulation, and optimization to diagnostics, design, and technica...

By A. Cunha Jr, C. Nataraj, L. M. Lorenzo, H. Panetto, A. Scotti, J. V. C. Vargas

agents safety

arXiv AI

Oct 27

NewsBench: Expert-Grounded Evaluation of Epistemic Quality in AI News Reporting

arXiv:2610.33240v1 Announce Type: new Abstract: As frontier AI systems increasingly serve as sources of political and current-events information, evaluating the epistemic quality of their responses i...

By Matt Wilde, Jillian Fisher, Andrew B. Hall, Kathryn Salam, Robbie Goldfarb

benchmarks safety

arXiv AI

Oct 27

A Comprehensive Framework for All Stages of Large-Model Training: From Data and Tokenization, through Pretraining, to Post-Training Alignment

arXiv:2610.33451v1 Announce Type: new Abstract: The capabilities of large language models (LLMs) and multimodal LLMs (MLLMs) arise from the synergy of the full training pipeline, including data and t...

By Xiangjing An, Zhibin Jiao, Aiting Chen

llms nlp reinforcement-learning fine-tuning multimodal safety data-engineering

arXiv AI

Oct 27

Auditing evaluation artifacts and data leakage in large language model systems for rare disease diagnosis

arXiv:2610.33592v1 Announce Type: new Abstract: Augmented large language model (LLM) systems have reported high benchmark accuracies in rare disease diagnosis, yet their real-world clinical utility r...

By Jian Yang, Peilin Liu, Xiaohan Li, Songda Li, Yu Cao, Yuqing Feng, Haomin Li

llms agents benchmarks safety

arXiv AI

Oct 27

Collective Emergent Alignment: Measuring and Amplifying Interaction-Induced Reasoning Alignment in Multi-Agent LLMs

arXiv:2610.34836v1 Announce Type: new Abstract: Multi-agent large language model (LLM) systems can exhibit emergent behaviors through interaction, yet whether a similar phenomenon exists for reasonin...

By Boxuan Wang, Zhuoyun Li, Sarvapali D. Ramchurn, Xiaowei Huang, Yi Dong

llms agents benchmarks safety

arXiv AI

Oct 27

DocTalk: Enhancing Medical Knowledge Retrieval via Fine-tuned Large Language Models with Chain-of-Thought Reasoning for Robust Clinical Decision Support

arXiv:2610.34871v1 Announce Type: new Abstract: Objective: The rapid advancement of large language models (LLMs) in healthcare has opened new possibilities for AI-assisted clinical decision-making; h...

By Asad Gohar, Md Ariful Islam Mozumder, Moon-Il Joo, Hee-Cheol Kim

llms fine-tuning safety

arXiv AI

Oct 27

C-SafeGen: Certified Safe LLM Generation with Claim-Based Streaming Guardrails

arXiv:2610.34961v1 Announce Type: new Abstract: Despite the remarkable capabilities of large language models (LLMs) across diverse applications, they remain vulnerable to generating content that viol...

By Mintong Kang, Zhaorun Chen, Bo Li

llms benchmarks safety

arXiv AI

Oct 27

Beyond Single-Attribute Auditing: A Configurable Multi-Attribute Benchmark for Intersectional Fairness in LLMs

arXiv:2610.35069v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly evaluated for fairness, yet most existing benchmarks audit one sensitive attribute at a time. This single...

By Zheng Zhang, Yifan Wu, Yuxiang Zhou, Qi Liu, Qingyang Mao, Ning Li, Weibo Gao, Qiyuan Shi, Zhenya Huang, Baosheng Yu

llms benchmarks safety

arXiv AI

Oct 27

MuSID-KD: Multigranularity Knowledge Distillation for Generative Recommendation with Semantic IDs

arXiv:2610.35218v1 Announce Type: new Abstract: Generative recommendation with semantic IDs (SIDs) represents each item as a sequence of discrete tokens and autoregressively predicts the next item's...

By Fengshi Zhang, Xiyan Ding, Shuxiao Xie, Wei Zhang

efficiency safety

arXiv AI

Oct 27

Trustworthy Recursive Self-Improvement in the Era of Large Models: Foundation, Risks, Evaluation and Open Challenges

arXiv:2610.35193v1 Announce Type: new Abstract: AI systems built around large models increasingly use experience and feedback to make persistent changes to models, prompts, memories, tools, and workf...

By Jiayue Pu, Ang Gao, Jinyi Bai, Qipeng Wang, Tongyu Wen, Jiahao Zhao, Keyu Qu, Junhao Zhang, Chenglei Shen, Zhongxiang Sun, Jun Xu, Xiao Zhang

safety

arXiv AI

Oct 27

AdaTrendLike: Adaptive Hierarchical Detrending and Trend Likelihood Scoring for Time Series Anomaly Detection

arXiv:2610.35400v1 Announce Type: new Abstract: The rapid proliferation of Industrial Internet of Things devices has led to increasingly large-scale and non-stationary time-series data, posing signif...

By Van Kwan Zhi Koh, Songnan Lin, Rongmo Luo, Zhiping Lin, Bihan Wen

rag benchmarks safety

arXiv AI

Oct 27

CANS: Capability-Preserving Neuron Steering for Large Language Models

arXiv:2610.35530v1 Announce Type: new Abstract: Activation steering offers training-free and dynamic control of target behaviors in large language models, but existing methods can impair general capa...

By Yucheng Wu, Chengyi Xie, Chenpeng Wang, Liangming Pan

llms fine-tuning benchmarks safety

arXiv AI

Oct 27

Aerial-R1: Reinforcing Aerial Reasoning and Segmentation in MLLMs

arXiv:2610.33056v1 Announce Type: cross Abstract: While Multimodal Large Language Models (MLLMs) have revolutionized general visual perception, existing methods suffer from severe hallucination in th...

By Jisheng Dang, Huaiyuan Peng, Yixuan Wang, Kun Wang, Zhang Li, Tat-Seng Chua

llms agents computer-vision fine-tuning multimodal benchmarks safety

arXiv AI

Oct 27

MOSAIC: Towards a Universal Latent Space for Computational Pathology Foundation Models

arXiv:2610.33059v1 Announce Type: cross Abstract: Pathology foundation models (PFMs) differ substantially in architecture, pretraining objective, training data and representation dimensionality, yet...

By Dibaloke Chanda, Praveen Madiraju

diffusion benchmarks safety

arXiv AI

Oct 27

When Agents Collude: Weak-to-Strong On-Policy Distillation for Mitigating Strategic Deception in Multi-Agent Systems

arXiv:2610.33011v1 Announce Type: cross Abstract: While recent advances in autonomous agents enable them to coordinate on increasingly complex tasks, prior work shows that this growing capacity for c...

By Eric Onyame, Runtao Zhou, Kowshik Thopalli, Bhavya Kailkhura, Chirag Agarwal

llms agents efficiency safety

arXiv AI

Oct 27

C-DFF: Concept-Level Diagnostics for Mechanism Parity in Simulation-Based Validation of Autonomous Driving Perception

arXiv:2610.33760v1 Announce Type: cross Abstract: Simulation is widely used in autonomous-driving validation, but matching task outputs between real and synthetic images does not guarantee that a sys...

By Danial Safaei, Siddartha Khastgir, Matthew Higgins, Jeroen Ploeg, Mohsen Alirezaei, Son Tong, Ludo Gielen

agents computer-vision robotics safety

Get the daily briefing