AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

10,632 stories · RSS feed

arXiv Machine Learning

Jun 26

Quantization in Federated Learning: Methods, Challenges and Future Directions

arXiv:2606. 26822v1 Announce Type: new Abstract: Federated Learning (FL) has become a foundational paradigm for privacy-preserving distributed intelligence, yet its scalability remains fundamentally constrained by communication bottlenecks, device heterogeneity, and the challenges of training under statistically non-IID data.

By Farwa Ikram, Dipanwita Thakur, Antonella Guzzo, Giancarlo Fortino

efficiency safety

arXiv Machine Learning

Jun 26

Sample-efficient Transfer Reinforcement Learning via Adaptive Reward Shaping and Policy-Ratio Reweighting Strategy

arXiv:2606. 26527v1 Announce Type: new Abstract: Transfer learning improves policy learning efficiency by reusing knowledge from source tasks, providing a feasible paradigm for safe and efficient autonomous highway lane changing decision-making.

By Wenjie Huang, Yang Li, Jingjia Teng, Mingwei Jin, Kai Song, Yougang Bian, Yongfu Li, Qisong Yang, Helai Huang

agents reinforcement-learning fine-tuning safety

arXiv AI

Jun 26

Reconstruction Alignment Improves Unified Multimodal Models

arXiv:2509. 07295v4 Announce Type: replace-cross Abstract: Unified multimodal models (UMMs) unify visual understanding and generation within a single architecture.

By Ji Xie, Trevor Darrell, Luke Zettlemoyer, XuDong Wang

rag diffusion multimodal benchmarks safety

arXiv Machine Learning

Jun 26

Can Large Language Models Reliably Code Qualitative Humanitarian Data? A Benchmark Study Against Human Expert Adjudication

arXiv:2606. 26541v1 Announce Type: new Abstract: Data from affected populations are crucial for informing humanitarian response, but their value depends on timely and consistent interpretation of nuanced accounts of need.

By Jerome Marston, Tino Kreutzer, Salom\'e Garnier, Ella Boone, Phuong N Pham, Patrick Vinck

llms benchmarks safety

arXiv Machine Learning

Jun 26

Learning Probabilistic Filters with Strictly Proper Scoring Rules

arXiv:2606. 26497v1 Announce Type: new Abstract: Bayesian filtering of partially and noisily observed dynamical systems seeks to infer the evolving conditional distribution of the state of a dynamical system, given observations, in an online fashion.

By Eviatar Bach, Ricardo Baptista, Jochen Br\"ocker, Bohan Chen, Andrew Stuart

llms multimodal safety

arXiv Machine Learning

Jun 26

High-Probability PL-SGD with Markovian Noise: Optimal Mixing and Tail Dependence

arXiv:2606. 26316v1 Announce Type: new Abstract: We study first-order methods for smooth objectives satisfying the Polyak-\L{}ojasiewicz (PL) condition when gradient samples are generated by an exogenous Markov chain.

By Dhruv Sarkar, Aprameyo Chakrabartty, Vaneet Aggarwal

safety

arXiv Machine Learning

Jun 26

Graph Neural Networks Applications Across Domains: All Insights You Need

arXiv:2606. 27202v1 Announce Type: new Abstract: Graph neural networks have moved from a niche representation-learning technique to the default model class wherever data carry relational structure.

By Abderaouf Bahi

computer-vision safety

arXiv Machine Learning

Jun 26

RecallRisk-BERT: A Multi-Task Framework for Post-Report Medical Device Recall Triage

arXiv:2606. 27174v1 Announce Type: new Abstract: Medical device recalls are a critical regulatory mechanism for protecting patient safety.

By Ali Semih Atalay, Sevgi Yigit-Sert

rag safety

arXiv Machine Learning

Jun 26

Implementation of reinforcement learning in chemical reaction networks: application to phototaxis as curiosity-driven exploration

arXiv:2606. 26168v1 Announce Type: new Abstract: Living systems navigate environments using noisy and incomplete sensory signals.

By Ruyi Tang (LCQB-AG), Gr\'egoire Sergeant-Perthuis (LCQB-AG), David Colliaux

reinforcement-learning benchmarks safety

arXiv Machine Learning

Jun 26

Necessary but Not Sufficient: Temperature Control and Reproducibility in LLM-as-Judge Safety Evaluations

arXiv:2606. 26185v1 Announce Type: new Abstract: LLM-as-judge ("grader") components are now standard in evaluation harnesses, including safety evaluations where a pass/fail verdict may gate downstream deployment decisions.

By Hiroki Tamba

llms safety

arXiv AI

Jun 26

MIRROR: Novelty-Constrained Memory-Guided MCTS Red-Teaming for Agentic RAG

arXiv:2606. 26793v1 Announce Type: cross Abstract: Multimodal agentic retrieval-augmented generation (RAG) systems expand the attack surface beyond prompt injection to include text poisoning, image injection, direct-query attacks, and orchestrator-level tool manipulation.

By Inderjeet Singh, Andr\'es Murillo, Motoyoshi Sekiya, Yuki Unno, Junichi Suga

rag agents robotics multimodal benchmarks safety

arXiv Machine Learning

Jun 26

The Geometry of Updates: Fisher Alignment at Vocabulary Scale

arXiv:2606. 27242v1 Announce Type: new Abstract: Training-free source selection for LLM families with shared vocabularies arises in scientific string domains such as SMILES, protein, and genomic sequences, where candidate corpora share a tokenizer but differ in prediction targets.

By John Sweeney

llms rag nlp safety

arXiv Machine Learning

Jun 26

Designing Reward Signals for Portable Query Generation: A Case Study in Industrial Semantic Job Search

arXiv:2606. 27291v1 Announce Type: new Abstract: Job-search platforms rely on low-bandwidth query interfaces that often fail to capture the high-dimensional complexity of candidate profiles.

By Ping Liu, Qianqi Shen, Jianqiang Shen, Wenqiong Liu, Rajat Arora, Yunxiang Ren, Chunnan Yao, Dan Xu, Baofen Zheng, Wanjun Jiang, Andrii Soviak, Kevin Kao, Jingwei Wu, Wenjing Zhang

llms reinforcement-learning safety

arXiv AI

Jun 26

Risk-Aware Selective Multimodal Driver Monitoring with Driver-State World Modeling

arXiv:2606. 26922v1 Announce Type: cross Abstract: Continuous driver monitoring in automated vehicles requires low-latency inference while avoiding unsafe decisions under uncertain driver states.

By Daosheng Qiu, Haozhuang Chi, Hao Su, Shu Long, Xinyue Miao, Yongle Dong, Wei Zhang

llms efficiency multimodal safety

arXiv Machine Learning

Jun 26

Hallucination in World Models is Predictable and Preventable

arXiv:2606. 27326v1 Announce Type: new Abstract: Modern generative world models render increasingly realistic action-controllable futures, yet they frequently hallucinate: rollouts remain visually fluent while drifting from the ground-truth dynamics.

By Nicklas Hansen, Xiaolong Wang

fine-tuning safety

arXiv AI

Jun 26

Symbolic Reasoning Frameworks Trigger Memory-Mediated Ecosystem Dynamics in Multi-Agent LLM Systems

arXiv:2606. 07552v2 Announce Type: replace-cross Abstract: Large language models exhibit a risk-averse "turtle" bias as strategic agents.

By Augustin Chan

llms agents benchmarks safety

arXiv AI

Jun 26

Bridging Vision and Language Concepts through Optimal Transport Semantic Flow

arXiv:2606. 26891v1 Announce Type: cross Abstract: Concept Bottleneck Models (CBMs) promise transparent reasoning by predicting through human-interpretable concepts, yet their effectiveness fundamentally depends on how well visual and textual representations are aligned or matched.

By Chenyang Zhang, Anqi Dong, Guangming Zhu, Nuoye Xiong, Siyuan Wang, Lin Mei, Liang Zhang

diffusion multimodal safety

arXiv AI

Jun 26

Boundary-Aware Context Grounding for A Low-Channel EEG Agent

arXiv:2606. 26519v1 Announce Type: new Abstract: Large language models (LLMs) can make scientific software easier to use.

By Zhiyuan Xu, Yueqing Dai, Junling Li, Junwen Luo

llms agents benchmarks safety

arXiv AI

Jun 26

MLFFM-SegDiff: A Multi-Level Feature Fusion Diffusion Model for Skin Lesion Segmentation

arXiv:2606. 26712v1 Announce Type: cross Abstract: Skin lesion segmentation is a key task in computer-aided dermatological diagnosis, where accuracy directly impacts downstream analysis and disease classification.

By Jingjun Gu, Chaojie Shen, Yifeng Cao, Wei Zhang, Yiliu Li, Aobo Fan

diffusion computer-vision safety

arXiv AI

Jun 26

Benchmarking Open-Weight Foundation Models for Global AI Technical Governance

arXiv:2606. 26099v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly deployed in artificial intelligence (AI) governance analysis across national and international organisations.

By Jason Hung

llms benchmarks safety

Get the daily briefing