AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

8,629 stories · RSS feed

arXiv AI

Aug 10

TOFD: Target-Oriented Feature Decoupling against Poisoning Attacks in Split Federated Learning

arXiv:2608. 07274v1 Announce Type: cross Abstract: Split Federated Learning (SFL) facilitates privacy-preserving collaborative training with reduced client-side overhead.

By Yuhan Xie, Jingrong Huang, Chen Lyu

benchmarks safety

arXiv Machine Learning

Aug 10

Online Conformal Prediction Beyond Feedback

arXiv:2608. 07139v1 Announce Type: new Abstract: Uncertainty quantification is essential when deploying machine learning models in safety-critical applications.

By Joar Skalse, Edoardo Pona, Osvaldo Simeone, Nicola Paoletti

reinforcement-learning safety

arXiv AI

Aug 10

LSEAD: A Privacy-Preserving LLM-Based Speech Analysis Framework for Early Alzheimer's Disease Screening

arXiv:2608. 07378v1 Announce Type: cross Abstract: Early diagnosis of Alzheimer's disease (AD) is critical for enabling timely interventions that may slow disease progression and improve patient outcomes.

By Xin Wang, Yingchao Huang, Yuhan Su, Shanshan Yao, Wei Peng

llms rag benchmarks safety

arXiv AI

Aug 10

ResidencyRL: Reinforcement Learning in Simulated Clinical Environments

arXiv:2608. 07418v1 Announce Type: new Abstract: In medical education, physicians convert academic knowledge into clinical expertise through residency: years of training across thousands of encounters, with diverse sources of feedback and progressively greater autonomy.

By Valentin Li\'{e}vin, Samuel Schmidgall, Tim Strother, Alex Bijamov, Akshay Goel, Anil Palepu, Chunjong Park, Vahid Balazadeh, Min Woo Sun, Marius Guerard, Justin Chen, Dave Steiner, Vikram Dhillon, Ibrahim Azar, Akhil Mehta, Nicholas Spetsieris, Shilpan Shah, Maen Abdelrahim, Amit Dahiya, Yun Liu, Katherine Chou, Yossi Matias, Avinatan Hassidim, Dale R. Webster, Quoc V. Le, Raia Hadsell, Joelle Barral, Carey Radebaugh, Aleksandra Faust, Shekoofeh Azizi, Mike Schaekermann, Po-Hsuan Cameron Chen, Tao Tu, David Racz, Lin Yang

llms agents reinforcement-learning benchmarks safety

arXiv Machine Learning

Aug 10

Let's Unlearn Stereotypes Before Decision-Making: Assessing the Impact of Intrinsic Bias Mitigation on Downstream Fairness in LLMs

arXiv:2509. 16462v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) are increasingly used in high-stakes decision-making systems, where biased predictions can reinforce social and economic disparities.

By Mina Arzaghi, Alireza Dehghanpour Farashah, Florian Carichon, Jean-Fran\c{c}ois Plante, Golnoosh Farnadi

llms rag fine-tuning benchmarks safety

arXiv AI

Aug 10

Beyond Foundation Models: Dimension-Aware Neural Architecture Search with Small-Data Representation Models for Cryocooler Lifetime Prediction

arXiv:2608. 06993v1 Announce Type: cross Abstract: Large-scale pretrained time-series models achieve strong results through large-scale pretraining and task-agnostic representation learning, but they rely on abundant, diverse data that industrial and scientific domains often lack.

By Gregor Molan (Comtrade 360 d.o.o., Letali\v{s}ka cesta 29b, Ljubljana, 1000, Slovenia), Grafika Jati (Comtrade 360 d.o.o., Letali\v{s}ka cesta 29b, Ljubljana, 1000, Slovenia), Francesco Barchi (Alma Mater Studiorum - Universita di Bologna, Department of Electrical, Electronic, and Information Engineering), Andrea Acquaviva (Alma Mater Studiorum - Universita di Bologna, Department of Electrical, Electronic, and Information Engineering), Alja\v{z} Osterman (LE-Tehnika d.o.o., \v{S}uceva 27, Kranj, 4000, Slovenia), Martin Molan (Comtrade AI GmbH, Grafenauweg 8, Zug, 6300, Switzerland)

llms safety

arXiv AI

Aug 10

A MARL Centered Reference Architecture for Large Language Model Augmentation in Smart Manufacturing

arXiv:2608. 07148v1 Announce Type: new Abstract: Modern manufacturing imposes six coupled demands on adaptive control: local decisions with global consequences, partial observability, nonstationarity, reflex speed response with long horizon effects, delayed and diffuse outcomes, and dynamics that resist explicit modeling.

By Fouad Bahrpeyma, Dirk Reichelt

llms agents reinforcement-learning safety

arXiv AI

Aug 10

Transformers Struggle to Use Their Emergent World Models: Revisiting the Tower of Hanoi, and the Illusion of Thinking

arXiv:2608. 07077v1 Announce Type: new Abstract: The Tower of Hanoi is a simple planning puzzle that in prior work has proven challenging for large reasoning models (LRMs).

By Devin Pereira, Willem Zuidema

llms safety

arXiv Machine Learning

Aug 10

CHIME: A Case for Efficient Long-Context Attention-FC Disaggregated Inference with DIMM-PIM

arXiv:2504. 17584v2 Announce Type: replace-cross Abstract: Attention-FC Disaggregated (AFD) LLM inference systems offload memory-bound Attention operations to memory-rich accelerators (e.

By Qingyuan Liu, Liyan Chen, Haocheng Wang, Yanning Yang, Dong Du, Zhigang Mao, Naifeng Jing, Yubin Xia, Haibo Chen

llms efficiency benchmarks safety

arXiv AI

Aug 10

Post-Grokking Collapse at the Representation-Readout Interface in Muon-Trained Transformers

arXiv:2608. 07436v1 Announce Type: new Abstract: Under the standard split, Muon gets hidden matrices and AdamW embeddings/output head.

By Ali Janati, Kaoutar El Maghraoui, Andrei Kanavalau, Anass Belfatmi

llms rag safety

arXiv Machine Learning

Aug 10

SNI-GNN: SmartNIC-Assisted Full-Graph GNN Training with In-Network Embedding Prediction

arXiv:2608. 06441v1 Announce Type: new Abstract: Full-graph GNN training delivers high accuracy but scales poorly on multi-server clusters due to heavy, irregular inter-node embedding exchanges.

By Guofan Yu, Sitian Chen, Zhenheng Tang, Xiaowen Chu, Amelie Chi Zhou

rag benchmarks safety data-engineering

arXiv AI

Aug 10

CreativeInstruct: Scalably Teaching LLMs to Balance Quality, Creativity, and Diversity

arXiv:2608. 07460v1 Announce Type: cross Abstract: While post-training improves the capabilities of large language models (LLMs), it generally lowers their output diversity and creativity, negatively impacting tasks that explicitly require creativity (e.

By Ananya Sahu, Mohit Bansal, Elias Stengel-Eskin

llms reinforcement-learning safety

arXiv AI

Aug 10

Mind the Gap: A Dual Knowledge Graph Framework for Unified Multi-task User Intent Inference

arXiv:2608. 06752v1 Announce Type: new Abstract: This paper proposes DKG-MTI, a dual knowledge graph framework for unified multi-task user intent inference from online travel reviews.

By Tzu-Cheng Peng (National Taiwan University), Chien Chin Chen (National Taiwan University), Chih-Hao Ku (University of North Texas), Yung-Chun Chang (Taipei Medical University)

llms safety

arXiv AI

Aug 10

Divergent Response Modes in Frontier Language Models Under Steering Pressure

arXiv:2608. 06578v1 Announce Type: new Abstract: Frontier language models are trained using distinct data, objectives, and safety pipelines.

By Ali Jalal-Kamali

llms safety

arXiv AI

Aug 10

Finding Usable Weight Mechanisms with Tiled SVD

arXiv:2608. 06969v1 Announce Type: new Abstract: The dominant approach to mechanistic interpretability trains proxy dictionaries such as sparse autoencoders and labels features from max-activating text.

By Ash Manvi, Samreena Tajreen

safety

arXiv Machine Learning

Aug 10

FedDOSE: Federated Learning Framework Decomposing Site Effects for Modeling Brain Dynamic Functional Connectivity

arXiv:2608. 07393v1 Announce Type: new Abstract: Functional Magnetic Resonance Imaging ( fMRI ) data are often pooled into collaborative multi-site consortia, as deep learning models for analyses require large datasets to generalize well.

By Deepank Girish, Yi Hao Chan, Yubin Zheng, Sukrit Gupta, Jagath C. Rajapakse

benchmarks safety

arXiv AI

Aug 10

Multi-Legal-Bench: Evaluating LLMs on Legal Reasoning Across Jurisdictions, Languages, and Legal Traditions

arXiv:2605. 29738v2 Announce Type: replace-cross Abstract: Legal NLP benchmarks overwhelmingly evaluate a single language or aggregate tasks that differ fundamentally across jurisdictions, making cross-lingual comparison impossible.

By Volodymyr Ovcharov

llms nlp benchmarks safety

arXiv Machine Learning

Aug 10

Game-Theoretic Inverse Reinforcement Learning for Modeling Competitive Human Driving: A Cut-in Prediction Study

arXiv:2608. 06445v1 Announce Type: cross Abstract: Capturing the strategic decision-making inherent in competitive human driving is critical for autonomous vehicle safety and traffic simulation.

By Yu Song

agents reinforcement-learning benchmarks safety

arXiv AI

Aug 10

Trustworthy Agent Network: Trust in Agent Networks Must Be Baked In, Not Bolted On

arXiv:2605. 19035v2 Announce Type: replace Abstract: The rapid advancement of Large Language Models has given rise to autonomous LLM-based agents capable of complex reasoning and execution.

By Yixiang Yao, Yuhang Yao, Xinyi Fan, Jiechao Gao, Jie Wang, Minjia Zhang, Srivatsan Ravi, Carlee Joe-Wong

llms agents safety

arXiv AI

Aug 10

Long-Horizon Agent Trajectory Attribution: A Unified Benchmark and Fine-Grained Annotation Framework

arXiv:2608. 06909v1 Announce Type: new Abstract: Large language model (LLM) agents increasingly operate through long-horizon trajectories involving user instructions, tool use, external observations, and memory.

By Jing Chen, Yang Sun, Li Zhang, Lin Xu, Jie Shi

llms agents benchmarks safety

Get the daily briefing