AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

9,037 stories · RSS feed

arXiv Machine Learning

Jul 31

DAS-PMVC: A Framework for Partial Multi-View Clustering via Dual Alignment and Structure Enhancement

arXiv:2607. 27761v1 Announce Type: new Abstract: In recent years, multi-view clustering has attracted widespread research interest.

By Shubin Ma, Liang Zhao, Chuanye He, Zhenjiao Liu, Liang Zou, Lin Yuanbo Wu, Yu Shao

rag diffusion benchmarks safety

arXiv Machine Learning

Jul 31

Beyond the Bidirectional Promise: Re-evaluating the Robustness of Diffusion Language Models

arXiv:2607. 27386v1 Announce Type: cross Abstract: Diffusion Language Models (DLMs) offer a compelling alternative to autoregressive (AR) generation by enabling bidirectional context and iterative refinement.

By Saurabh Yadav, Badri Narayana Patro, Vijay Srinivas Agneeswaran

llms diffusion safety

arXiv Machine Learning

Jul 31

Harnessing the Potential of Optimizing Data Mixtures via Bayesian Domain Reweighting

arXiv:2607. 27928v1 Announce Type: new Abstract: The performance of Large Language Models (LLMs) is fundamentally influenced by the distributional composition of multi-domain pre-training data.

By Xiang Yuan, Kaiqing Lei, Zhenyu Jin, Jun Shu, Deyu Meng, Zongben Xu

llms safety

arXiv Machine Learning

Jul 31

On-Policy and Off-Policy Learning for Large Action Spaces

arXiv:2607. 28408v1 Announce Type: new Abstract: This thesis studies policy learning in interactive systems where an agent observes a context, selects an action from a very large set, and receives partial feedback.

By Imad Aouali

agents diffusion safety

arXiv Machine Learning

Jul 31

Heterogeneous Ranking in Industrial-Scale Recommender Systems: A Case Study

arXiv:2607. 27577v1 Announce Type: cross Abstract: Heterogeneous recommendation feeds present complex challenges that extend beyond those found in highly homogeneous environments (e.

By Di Bai, Jintao Liu, Zhenwei Tang, Peifan Wu, Nada Al-Thawr, Luoshu Wang

benchmarks safety

arXiv Machine Learning

Jul 31

Dynamic Spectral Filtering for Temporal Graph Learning: Learning Evolving Propagation Operators

arXiv:2607. 27891v1 Announce Type: cross Abstract: Temporal graph learning is commonly organized around the evolution of node states or the encoding of interaction histories.

By Yan Kong

benchmarks safety

arXiv Machine Learning

Jul 31

Integrating Contextual Embeddings into Evaluation of Expressive MIDI Piano Performances

arXiv:2607. 27909v1 Announce Type: cross Abstract: Objective evaluation of expressive MIDI piano performances typically relies on attribute statistics such as timing, velocity, and duration of individual notes.

By Dmitrii Gavrilev, Ilya Borovik, Vladimir Viro

rag safety

arXiv Machine Learning

Jul 31

Safety-Gated Agentic Supervisory Control on a Coupled Distillation Benchmark: Regime Map, Auditable Gate, and Co-Design Findings

arXiv:2607. 27849v1 Announce Type: cross Abstract: An open-weight LLM can write composition setpoints every five minutes.

By Christian Rosenthal

llms agents efficiency benchmarks safety

arXiv AI

Jul 31

AI Security Priorities: A Field-Wide Agenda

arXiv:2607. 26069v1 Announce Type: cross Abstract: As AI systems are rapidly integrated into critical economic, governmental, and national security functions, the gap between AI adoption and AI security readiness continues to widen.

By Gil Gekker, Rachel Steratore, Everett Smith, Asher Brass-Gershovich, Varun Gandhi, Nicole Nichols, Vijay Bolina, Buck Shlegeris, Lisa Einstein, Dan Lahav, Omer Nevo, Sella Nevo

agents safety

arXiv AI

Jul 31

The Human Utility Factor: A Computable Welfare Metric That Reframes AI Governance as a Constrained Optimisation Problem

arXiv:2607. 26068v1 Announce Type: cross Abstract: Existing AI governance frameworks, including the EU AI Act and NIST AI RMF, address safety, transparency, and accountability but do not operationalize quantitative constraints on macro-socioeconomic stability.

By Sivasathivel Kandasamy

agents reinforcement-learning safety

arXiv AI

Jul 31

Optimizing Sensor Placement for Hydrogen Leak Detection in Enclosed Infrastructure: A Comparative Study Using CFD-informed Genetic Algorithm and DeepSets Neural Surrogate

arXiv:2607. 26078v1 Announce Type: cross Abstract: Hydrogen infrastructure in enclosed environments, such as parking facilities for fuel cell vehicles, presents significant safety challenges due to hydrogen's low ignition energy and wide flammability range.

By Fangnian Wang, Nicholas Tan Jerome, Thomas Jordan, Frank Simon

safety

arXiv AI

Jul 31

The Easy Trap: Why LLMs Underestimate Misconception-Driven Difficulty

arXiv:2607. 26067v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used for estimating item difficulty in educational assessment.

By Amanda La Hadi, Muhammad Johan Alibasa, Guanliang Chen, A. Taufiq Asyhari

llms safety

arXiv AI

Jul 31

Identifying Implicit Bias in LLM-based Chat AI Toward People with Intellectual Disabilities

arXiv:2607. 26062v1 Announce Type: cross Abstract: Background: This work investigates the presence of implicit bias in Large Language Model (LLM)-based chat AI models directed toward people with intellectual disabilities (ID).

By Karly V. Coffey, Gloria L. Krahn, John P. Hanley, Jacob E. Neely

llms safety

arXiv AI

Jul 31

OmegaUse-OfficeVal: Benchmarking LLM Agents on Long-Horizon Office-Suite Tasks with Economic Grounding

arXiv:2607. 27155v1 Announce Type: new Abstract: Large language model (LLM) agents are increasingly expected to assist users in completing tasks.

By Jingbo Zhou, Yusai Zhao, Qi Bao, Jingjia Cao, Zhenghai Chen, Chang Gao, Kaiqi Guo, Muxin Guo, Mingxuan Li, Xinjiang Lu, Yanru Ma, Yixiong Xiao, Zenghui Zhang, Le Zhang, Hua Wu

llms agents benchmarks safety

arXiv Machine Learning

Jul 31

Generalization and Trade-off in Adversarial Training: An RKHS Perspective via Kernel Integral Operators

arXiv:2607. 27995v1 Announce Type: cross Abstract: Adversarial training has emerged as a powerful approach for protecting models against adversarial attacks in a broad range of real-world applications.

By Yiling Xie, Xiaoming Huo

benchmarks safety

arXiv AI

Jul 31

Even More Deception: Objective Misalignment in Mixed-Motive LLM Multi-Agent Systems

arXiv:2607. 26120v1 Announce Type: new Abstract: Large Language Models (LLMs)-powered multi-agent systems are increasingly deployed in mixed-motive environments, where agents operate under asymmetric information and strategic deception due to conflicting or hidden objectives.

By Marylou Fauchard, Florian Carichon, Margarida Carvalho, Golnoosh Farnadi

llms agents safety

arXiv Machine Learning

Jul 31

Uncertainty quantification for trustworthy deep learning: Methods and measures

arXiv:2607. 28248v1 Announce Type: cross Abstract: The deployment of deep neural networks in safety-critical domains demands reliable estimates of predictive confidence, yet conventional architectures lack principled uncertainty quantification.

By H. Martin Gillis, Thomas Trappenberg

llms safety

arXiv Machine Learning

Jul 31

Robust Estimation of Sparse Numerical Vectors under Local Differential Privacy

arXiv:2607. 27815v1 Announce Type: cross Abstract: Local differential privacy (LDP) protocols are vulnerable to poisoning attacks.

By Puning Zhao, Zhikun Zhang, Shaowei Wang, Sheng Yue, Bangzhou Xin, Tianhang Zheng, Pengfei Zhang, Xiaochun Cao

safety

arXiv AI

Jul 31

Misalignment Has a Personality: A Big Five Account of Emergent Misalignment

arXiv:2607. 26389v1 Announce Type: cross Abstract: Fine-tuning a language model on data containing a narrow flaw, such as insecure code or incorrect mathematical answers, can cause broad misalignment through a mechanism that remains debated.

By Hasibur Rahman, Smit Desai

llms fine-tuning safety

arXiv Machine Learning

Jul 31

Gradient-free Task-Conditioned Retrieval for On-Device In-Context Learning

arXiv:2607. 27766v1 Announce Type: cross Abstract: On-device in-context learning (ICL) relies on pre-inference retrieval to select demonstrations for useful context before downstream model inference.

By Xinyu Luo, Hui Liu, Yihua Shao, Junyi Yang, Arindam Basu, Haoliang Li

llms fine-tuning multimodal benchmarks safety

Get the daily briefing