AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,896 stories · RSS feed

arXiv Machine Learning

Aug 10

Defining Energy Indicators for Impact Identification on Aerospace Composites: A Structured Feature Selection Approach Guided by Domain Knowledge

arXiv:2511. 01592v2 Announce Type: replace Abstract: Energy estimation is critical to impact identification on aerospace composites, where low-velocity impacts can induce internal damage that is undetectable at the surface.

By Nat\'alia Ribeiro Marinho, Richard Loendersloot, Frank Grooteman, Jan Willem Wiegman, Uraz Odyurt, Tiedo Tinga

rag efficiency safety

arXiv Machine Learning

Aug 10

Sharding Prevents LLM Oversight Failures and Adversarial Exploitation

arXiv:2608. 06422v1 Announce Type: new Abstract: Giving an LLM judge more compute does not necessarily make it check more requirements.

By Victor Akinwande, J. Zico Kolter, Aran Nayebi

llms safety

arXiv AI

Aug 10

Automated item evaluation: Predicting item acceptance and rejection using LLM-generated critiques

arXiv:2608. 06609v1 Announce Type: new Abstract: Automated item evaluation (AIE) refers to the use of computational methods to assess item quality without requiring manual expert review or field testing of the items under evaluation.

By Hotaka Maeda, Yikai Lu

llms fine-tuning safety

arXiv AI

Aug 10

Shape Your Feed: An LLM-based Agentic System for Conversational Recommendation

arXiv:2608. 06632v1 Announce Type: new Abstract: Industrial recommendation systems predominantly adopt a passive ranking paradigm that infers user preferences from implicit behavioral signals (e.

By Ziyun Xu, Bosen Ding, Yue Zhang, Ji Qi, Qingyuan Song, Jizhou Huang, Liwei Wang, Jefferey Santelli, Yue Weng, Qichao Que, Zhenheng Yang, Junfeng Pan, Linhong Zhu

llms agents reinforcement-learning efficiency multimodal safety

arXiv Machine Learning

Aug 10

Recovering Lesion Parameters from Aphasic Picture Naming Error Profiles in Large Language Models

arXiv:2608. 06429v1 Announce Type: cross Abstract: Interpretability methods for large language models (LLMs) describe internal state but do not directly test whether that state is causally sufficient to produce the observed behavior.

By Yong Yang, Roger Newman-Norlund, Xiang Guan, Saeed Ahmadi, Regan Willis, Nadra Salman, Kalil Warren, Sophie Arheix-Parras, Srihari Nelakuditi, Leonardo Bonilha, Christopher Rorden, Rutvik H. Desai, Julius Fridriksson

llms safety

arXiv Machine Learning

Aug 10

Density-Functional Excited-State Gradients and Nonadiabatic Couplings on a Consumer GPU from a Contraction-DAG

arXiv:2608. 06536v1 Announce Type: cross Abstract: Nonadiabatic dynamics needs an excited-state gradient and an interstate nonadiabatic coupling matrix element (NACME) at every nuclear geometry, and a double-hybrid functional's accuracy has been unavailable for the coupling.

By Rub\'en Dar\'io Guerrero

safety

arXiv Machine Learning

Aug 10

Fairis: Fairness-Aware Aggregation with Provable Influence Containment against Fairness Poisoning Attacks in Collaborative Machine Learning

arXiv:2608. 06469v1 Announce Type: cross Abstract: Collaborative machine learning among financial institutions must be both group-fair and robust against deliberate adversarial manipulation.

By Devharsh Trivedi, Nesrine Kaaniche, Nikos Triandopoulos, Maryline Laurent, Jackson Walters

robotics benchmarks safety

arXiv Machine Learning

Aug 10

Stream Learning: Partition-Fair Gossip Learning Without Tokens

arXiv:2608. 06946v1 Announce Type: cross Abstract: In gossip learning, a network of nodes trains a shared model collaboratively, without a central coordinator, by repeatedly exchanging parts of their local models.

By Fabien Mathieu (NPA), Alexandre Pham (NPA), Maria Gradinariu Potop-Butucaru (NPA), S{\'e}bastien Tixeuil (IUF, NPA)

benchmarks safety

arXiv AI

Aug 10

Mitigating Scoring Bias in LLM-as-a-Judge via Random Number Generation

arXiv:2608. 05726v1 Announce Type: cross Abstract: Large Language Models (LLMs) are often used as evaluators of text quality, known as LLM-as-a-Judge, which can outperform conventional automatic evaluation metrics that rely on reference texts.

By Yuma Asato, Kiyoaki Shirai, Natthawut Kertkeidkachorn

llms nlp safety

arXiv Machine Learning

Aug 10

Iterative Training of Physics-Informed Neural Networks with Fourier-enhanced Features

arXiv:2510. 19399v2 Announce Type: replace Abstract: Spectral bias, the tendency of neural networks to learn low-frequency features first, is a well-known issue with many training algorithms for physics-informed neural networks (PINNs).

By Yulun Wu, Miguel Aguiar, Karl H. Johansson, Matthieu Barreau

diffusion benchmarks safety

arXiv AI

Aug 10

Interpretable Unsupervised Community Detection with LLM-Symbolized Structured Processes

arXiv:2608. 06402v1 Announce Type: new Abstract: Community detection is a fundamental task in graph analytics that aims to identify cohesive groups of entities with similar behaviors or interests.

By Aoting Zeng, Kai Wang, Jianwei Wang, Yuxiang Sun, Yizhang He, Wenjie Zhang

llms benchmarks safety

arXiv AI

Aug 10

Alignment has a Fantasia Problem

arXiv:2604. 21827v2 Announce Type: replace Abstract: In accomplishing complex tasks, human cognition typically progresses from abstract to concrete (e.

By Nathanael Jo, Zoe De Simone, Mitchell Gordon, Ashia Wilson

benchmarks safety

arXiv AI

Aug 10

Minimal, Local, Causal Explanations for Jailbreak Success in Large Language Models

arXiv:2605. 00123v3 Announce Type: replace Abstract: Safety trained large language models (LLMs) can often be induced to answer harmful requests through jailbreak prompts.

By Shubham Kumar, Narendra Ahuja

llms benchmarks safety

arXiv AI

Aug 10

HarnessSafe: Evaluating Safety Across Persistent Carriers in Agent Harnesses

arXiv:2608. 06984v1 Announce Type: cross Abstract: Modern agent harnesses persist state across tasks and sessions through persistent carriers like memory, skills, tools, and shared artifacts.

By Xiao Zhang, Yusheng Wang, Yuhao Fei, Dongyuan Li, Zian Liang, Liuyu Xiang, Hongxun Gu, Zhaofeng He

agents benchmarks safety

arXiv AI

Aug 10

Beyond Routing Weights: Faithful Response-Level Interpretation of Mixture-of-Experts Reward Models via Contribution Contrast

arXiv:2608. 06400v1 Announce Type: new Abstract: Reward models are central to learning from human preferences, yet identifying what drives their predictions remains challenging.

By Yifan Wang, Jinyi Mu, Mayank Jobanputra, Yu Wang, Soyoung Oh, Isabel Valera, Vera Demberg

reinforcement-learning safety

arXiv AI

Aug 10

ResidencyRL: Reinforcement Learning in Simulated Clinical Environments

arXiv:2608. 07418v1 Announce Type: new Abstract: In medical education, physicians convert academic knowledge into clinical expertise through residency: years of training across thousands of encounters, with diverse sources of feedback and progressively greater autonomy.

By Valentin Li\'{e}vin, Samuel Schmidgall, Tim Strother, Alex Bijamov, Akshay Goel, Anil Palepu, Chunjong Park, Vahid Balazadeh, Min Woo Sun, Marius Guerard, Justin Chen, Dave Steiner, Vikram Dhillon, Ibrahim Azar, Akhil Mehta, Nicholas Spetsieris, Shilpan Shah, Maen Abdelrahim, Amit Dahiya, Yun Liu, Katherine Chou, Yossi Matias, Avinatan Hassidim, Dale R. Webster, Quoc V. Le, Raia Hadsell, Joelle Barral, Carey Radebaugh, Aleksandra Faust, Shekoofeh Azizi, Mike Schaekermann, Po-Hsuan Cameron Chen, Tao Tu, David Racz, Lin Yang

llms agents reinforcement-learning benchmarks safety

arXiv AI

Aug 10

LSEAD: A Privacy-Preserving LLM-Based Speech Analysis Framework for Early Alzheimer's Disease Screening

arXiv:2608. 07378v1 Announce Type: cross Abstract: Early diagnosis of Alzheimer's disease (AD) is critical for enabling timely interventions that may slow disease progression and improve patient outcomes.

By Xin Wang, Yingchao Huang, Yuhan Su, Shanshan Yao, Wei Peng

llms rag benchmarks safety

arXiv Machine Learning

Aug 10

Let's Unlearn Stereotypes Before Decision-Making: Assessing the Impact of Intrinsic Bias Mitigation on Downstream Fairness in LLMs

arXiv:2509. 16462v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) are increasingly used in high-stakes decision-making systems, where biased predictions can reinforce social and economic disparities.

By Mina Arzaghi, Alireza Dehghanpour Farashah, Florian Carichon, Jean-Fran\c{c}ois Plante, Golnoosh Farnadi

llms rag fine-tuning benchmarks safety

arXiv AI

Aug 10

Beyond Foundation Models: Dimension-Aware Neural Architecture Search with Small-Data Representation Models for Cryocooler Lifetime Prediction

arXiv:2608. 06993v1 Announce Type: cross Abstract: Large-scale pretrained time-series models achieve strong results through large-scale pretraining and task-agnostic representation learning, but they rely on abundant, diverse data that industrial and scientific domains often lack.

By Gregor Molan (Comtrade 360 d.o.o., Letali\v{s}ka cesta 29b, Ljubljana, 1000, Slovenia), Grafika Jati (Comtrade 360 d.o.o., Letali\v{s}ka cesta 29b, Ljubljana, 1000, Slovenia), Francesco Barchi (Alma Mater Studiorum - Universita di Bologna, Department of Electrical, Electronic, and Information Engineering), Andrea Acquaviva (Alma Mater Studiorum - Universita di Bologna, Department of Electrical, Electronic, and Information Engineering), Alja\v{z} Osterman (LE-Tehnika d.o.o., \v{S}uceva 27, Kranj, 4000, Slovenia), Martin Molan (Comtrade AI GmbH, Grafenauweg 8, Zug, 6300, Switzerland)

llms safety

arXiv Machine Learning

Aug 10

Equivariant Sparse Autoencoders: Mechanistic Interpretability of Neural Networks on Symmetric Data

arXiv:2511. 09432v2 Announce Type: replace Abstract: Machine learning (ML) models achieve remarkable performance but remain hard to interpret due to their scale and complexity.

By Ege Erdogan, Ana Lucic

safety

Get the daily briefing