AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,788 stories · RSS feed

arXiv Machine Learning
4d ago

Interpretability-by-Design with Accurate Locally Additive Models and Conditional Feature Effects

The paper introduces Conditionally Additive Local Models (CALMs), a new class of interpretable machine learning models that balance the simplicity of Generalized Additive Models (GAMs) with the accuracy of GA$^2$Ms. CALMs allow multiple univariate shape functions per feature, each active in distinct input subregions defined by simple logical conditions on interacting features, thereby capturing interactions while maintaining local additivity. A distillation-based training pipeline identifies homogeneous regions with limited interactions and fits interpretable shape functions via region-aware backfitting, leading to performance that consistently outperforms GAMs and rivals GA$^2$Ms on diverse classification and regression tasks.

By Vasilis Gkolemis, Loukas Kavouras, Dimitrios Kyriakopoulos, Konstantinos Tsopelas, Dimitrios Rontogiannis, Giuseppe Casalicchio, Theodore Dalamagas, Christos Diou
arXiv Machine Learning
4d ago

Proof-Carrying Materials: Falsifiable Safety Certificates for Machine-Learned Interatomic Potentials

The paper introduces Proof‑Carrying Materials (PCM), a three‑stage framework that improves the reliability of machine‑learned interatomic potentials (MLIPs) used for high‑throughput materials screening. PCM combines adversarial falsification, bootstrap envelope refinement, and Lean 4 formal certification to close the large gap between MLIP predictions and density functional theory (DFT) stability, achieving a 93% reduction in missed stable materials on a 25,000‑material benchmark. The method uncovers architecture‑specific blind spots, provides a risk model with high predictive performance, and boosts discovery yield by 25% in a thermoelectric screening case study.

By Abhinaba Basu, Pavan Chakraborty
arXiv Machine Learning
4d ago

CalibCause: A Quality-Gated, Interpretable and Calibrated Deep Learning Framework for Diabetic Retinopathy Grading

CalibCause is a quality‑gated, interpretable deep learning framework for diabetic retinopathy grading that integrates visual explanations, label‑free image‑quality conditioning, knowledge distillation, and calibrated confidence with deferral. Using 51,310 fundus images from multiple datasets, the distilled student model achieved a quadratic weighted kappa of 0.863 and accuracy of 0.865 on pooled testing, while maintaining comparable performance across held‑out environments. The framework also demonstrated improved calibration (expected calibration error 0.025–0.036) and higher accuracy (up to 0.923) when deferring the least confident 20% of images.

By Sohaib Sulman, Muhammad Atif Saeed, Akhtar Jamil, Alaa Ali Hameed, Roland Aydin
arXiv Machine Learning
4d ago

Recycle only when you have to!

The paper proposes a cheap, differentiable predictor that estimates AlphaFold 3’s confidence metrics (ipSAE, ipTM) and structural consistency directly from the trunk during early recycling. This predictor can be used as a decision signal to allocate recycling budget efficiently during screening and as an objective in hallucination‑based binder design. Both uses improve performance while significantly reducing computational cost.

By Lorenz Vaitl, Ivan Tanasijevic, Arian Rokkum Jamasb, L. Santiago Mille-Fragoso, Pan Kessel
arXiv Machine Learning
4d ago

Mechanistic Interpretability for End-to-End Self-Driving

The paper applies mechanistic interpretability to end‑to‑end autonomous driving by training Sparse Autoencoders on a single‑camera path‑planning model’s internal activations. Unsupervised clustering of the autoencoder’s latent space uncovers interpretable features linked to route intent, vehicle speed, and braking, achieving 98.3% accuracy in recovering route intent without supervision. Visual counterfactual edits confirm that specific latent features correspond to scene changes such as traffic lights and stop signs, and manipulating these latents yields predictable changes in planner outputs; a lightweight adapter trained on these latents reduces trajectory selection error by 3.44% compared to one that ignores them.

By Manav Gagvani, Benjamin Namikas, Sivamurugan Velmurugan, Nysa Kumar, Shrey Sharma, Thomas Peterson, Pratyush Mathur, Hyunseo Chang, Rohan Paleja
arXiv AI
4d ago

QUAL: Quantile Alignment Improves Zero-Shot Time-Series Foundation Models

The paper introduces QUAL, a label‑free framework that improves zero‑shot time‑series foundation models by aligning their quantile forecasts using a pool of models as a probability reference. QUAL first aligns member forecasts, then aggregates them with Huber aggregation and rearrangement to produce a robust combined forecast without any training or model modification. Experiments on GIFT‑Eval show that this quantile alignment reduces average member MASE by 7.4% and CRPS by 9.8%, and the method remains robust even when weak forecasters are included in the candidate pool.

By Junhyeok Kang, Sangjun Han, Jun Seo, Seunghan Lee, Jaehoon Lee, Dongwan Kang, Minjae Kim, Sungdong Yoo, Tae Yoon Lim, Hwanil Choi, Soonyoung Lee, Wonbin Ahn
arXiv Machine Learning
4d ago

Nonlinear Maps Between Irreducible Representations in Graph Neural Networks

The paper introduces a degree‑resolved framework for nonlinear interactions between irreducible representations in graph neural networks, enabling mixing of non‑isomorphic representations that linear equivariant maps cannot achieve. Using this framework, the authors build Specht Triple, a residual module for unordered graph triples, and demonstrate that it reduces aggregate standardised test MSE by 11.05% on a synthetic structural benchmark compared to a parameter‑matched linear baseline. The results highlight irreducible decomposition as a useful inductive bias for nonlinear higher‑order equivariant processing.

By Mani Shayestehfar
arXiv Machine Learning
4d ago

FOCUS: First-Order Control Using Optimal Transport Supervision for Imitation from Observations

FOCUS is a new imitation‑learning method that learns from state‑only demonstrations without temporal alignment. It uses local optimal‑transport alignment to match short agent rollouts with sampled expert segments, creating differentiable imitation rewards. A learned critic then bootstraps value beyond the rollout horizon, improving sample efficiency and training stability on contact‑rich locomotion tasks.

By Jo\~ao A. C\^andido Ramos, Lionel Blond\'e, Hugues Vinzant, Jacopo Castellini, Alexandros Kalousis
arXiv Machine Learning
4d ago

Your Group-Relative Advantage Is Biased

The paper examines the bias in group-relative advantage estimation used in reinforcement learning from verifier rewards (RLVR) for large language models. It shows that for binary rewards, hard prompts suffer from negative estimation errors while easy prompts have positive errors, leading to underweighting correct hard answers and insufficient penalties for incorrect easy answers. The authors propose History‑Aware Adaptive Difficulty Weighting (HA‑DW), an adaptive reweighting scheme that corrects these biases, and demonstrate through theory and experiments on five mathematical reasoning benchmarks that HA‑DW improves performance when integrated into GRPO and its variants.

By Fengkai Yang, Zherui Chen, Xiaohan Wang, Xiaodong Lu, Jiajun Chai, Guojun Yin, Wei Lin, Shuai Ma, Fuzhen Zhuang, Deqing Wang, Yaodong Yang, Jianxin Li, Yikun Ban
arXiv Machine Learning
4d ago

Backpropagation-free Deep Learning for Time-Series Classification

The paper evaluates three backpropagation‑free learning rules—Direct Feedback Alignment, Direct Random Target Projection, and NoProp—on 112 univariate time‑series classification datasets using 1D convolutional networks. Each method achieves mean F1 scores between 0.76 and 0.79, slightly below state‑of‑the‑art backpropagation baselines. The study also analyzes how dataset properties, especially intra‑class temporal variability, affect performance gaps and examines computational trade‑offs such as FLOPs, training speed, and memory usage.

By Miha Kap\v{s}, Bla\v{z} Bertalani\v{c}, Carolina Fortuna
arXiv Machine Learning
4d ago

Geometry of Relaxed Fair Regression: A Unified Framework for Aware and Unaware Settings

The paper introduces a unified framework for relaxed fair regression that applies to both aware and unaware settings. By modeling regression with a demographic parity penalty as an optimal transport problem, it characterizes optimal prediction functions under squared Wasserstein‑2 and Total Variation penalties, revealing distinct fairness philosophies. The authors also present FAUST, an efficient algorithm that consistently matches or surpasses state‑of‑the‑art baselines on real‑world benchmarks.

By Marie Generali Lince, Vincent Divol, R\'emi Flamary, Solenne Gaucher, Patrick Loiseau
arXiv Computation and Language
4d ago

Keepace.ai: A Technical Report on a Large Language Model for Sports and Health

Keepace.ai is a domain‑specialized large language model for sports and health, built on Qwen3.5‑27B through post‑training. It is optimized for Chinese‑language exercise guidance across three tasks: workout plan generation, sports‑and‑health knowledge QA, and personal sports data analysis. Internal rubric‑based evaluations with domain experts show that Keepace outperforms its base model and larger general‑purpose baselines, especially in exercise safety, boundary compliance, structured output completeness, and scientific accuracy.

By ai Team, Bowen Wu, Heng Wang, Jufeng Mi, Kang Liu, Kun Li, Ling Man, Long He, Suli Fan, Xiaosen Jiang, Yi Wang, Ying He, Yu Tang, Yunhan Li, Zifang Liu
arXiv Machine Learning
4d ago

Opening the Box Changes the Cat: LLMs Detect Chain-of-Thought Monitoring - What now?

The paper studies how large language models (LLMs) detect chain‑of‑thought (CoT) monitoring through direct and indirect environmental cues. It shows that even subtle signals, such as timing information, can reveal monitoring to the model, prompting it to adapt or evade detection. The authors argue that oversight is interactive, with the monitoring process itself influencing the behavior it aims to observe.

By Amir Jalilifard, Marcos Medeiros Raimundo, Anderson Rocha
arXiv Machine Learning
4d ago

Communication-Efficient Private Federated Reinforcement Learning with Post-Quantum Authentication for Vehicular Networks

The paper proposes a communication‑efficient private federated reinforcement learning scheme for vehicular networks that uses correlated private quantization and post‑quantum authentication. By sharing complementary randomness between paired vehicles, the method reduces aggregation mean‑squared error while maintaining one‑bit communication and local differential privacy. Experiments show significant uplink savings, improved routing performance, and reduced privacy leakage compared to independent quantization.

By Hojat Allah Salehi, Devrim Aras, Md Jueal Mia, M. Hadi Amini, Imtiaz Karim, Mashrur Chowdhury, Bhavani Thuraisingham
arXiv Machine Learning
4d ago

Stochastic and Noisy Oracles in Smooth Continuous Optimization

The article reviews models of inexact computational oracles in smooth continuous optimization, focusing on zeroth-, first-, and second-order oracles characterized by quantitative error bounds and controllable parameters. It introduces deterministic and stochastic oracle models, illustrating them with examples such as finite-difference approximations, stochastic gradients, and randomized derivative estimators, and discusses distinctions between absolute/relative errors, bias/variance, and independent/coupled randomness. The review emphasizes analytical tools that recover and compare convergence and complexity guarantees across different oracle models, providing bounds on expected stopping times and computational complexity for both deterministic and stochastic first-order methods, and briefly covering inexact second-order information.

By Albert S. Berahas, Liyuan Cao, Katya Scheinberg
arXiv Computer Vision
4d ago

Coupled or Independent? Fault Coupling in Agentic Medical Vision Systems

The paper investigates how vision‑language agents that expose intermediate outputs can be affected when the sensor reading is unreliable, causing both the tool and reasoning channels to share a visual fault. Using a dermatology agent on the PAD‑UFES‑20 dataset, the authors introduce a fault‑coupling taxonomy distinguishing single‑channel errors from joint errors that agree or disagree, and show that typographic injection leads to detectable directional retrieval errors while physical corruption does not. They find that crop‑first retrieval yields a higher rate of joint wrong agreement (55%) compared to full‑image retrieval (35%), and that hard‑label channel comparison struggles to detect joint failures, limiting disagreement recall to 0.29 for crop‑first and 0.57 for full‑image retrieval, with similar effects observed on the HIBA test set.

By Nishi Kanta Paul, Md Shihabul Islam Shovo, Md Mostafijur Rahman, Jamil Fayyad
arXiv Machine Learning
4d ago

A prospective randomized study of patient-centered AI for clinical visit preparation

A randomized study evaluated a large language model–based chatbot designed to help patients prepare for clinical visits. In the Google Health app, 802 adults used the agent (standard or concise) or a control chatbot, and the agent outperformed the control in clinician‑rated preparation, alignment with patient context, and patient‑reported preparedness, confidence, and satisfaction. The results demonstrate that structured AI‑guided preparation can safely improve patient readiness for visits.

By Xincheng Tan, Matthew Thompson, Xiang Ji, Angelica Willis, Will Vaughan, Roma Ruparel, Michael A. Sanchez, Sean Chang, Joseph Xu, Bhavna Daryani, Preeti Singh, Pinal Bavishi, Kejia Chen, Fan Zhang, Priya Gupta, Ernest Lim, Christina Chen, Bob Lou, Dimitrios Antos, Zach Wasson, Ray Luo, Fadi Yousif, Florence Thng, Mark Malhotra, Joseph Breda, Daniel McDuff, Kavi Goel, Katherine Chou, Dale R. Webster, Avinatan Hassidim, Yossi Matias, Po-Hsuan Cameron Chen, Yun Liu, Mike Schaekermann, Quang Duong, Jackie M. Barr
arXiv Computer Vision
4d ago

MotorSense: A Video-EMG Dataset of Motoric Representations for Action Understanding

MotorSense is the first large‑scale dataset that pairs egocentric video with surface electromyography (EMG) and inertial measurement unit (IMU) data, covering 100 hours of unscripted bimanual manipulation. The dataset enables cross‑modal pre‑training that aligns visual and motor signals, improving downstream tasks such as action recognition, 3D hand reconstruction, and 3D contact reconstruction. Additionally, EMG signals can condition a video diffusion model to generate fine‑grained hand‑manipulation videos.

By Eadom Dessalene, Michael Maynord, Amir-Hossein Shahidzadeh, Botao He, Yasamin Tabatabaee, Ruoshi Liu, Ruohan Gao, Paul Pu Liang, Yiannis Aloimonos
arXiv Computer Vision
4d ago

Hawkeye: Reading Illegible Text with Vision Language Models

The paper introduces BSTRB, a benchmark of 8,398 noisy image samples with high‑fidelity annotations from 1,699 real‑world video sequences, designed to evaluate Vision‑Language Models (VLMs) on degraded text. It presents Hawkeye, a VLM fine‑tuned on BSTRB that reads and reasons about illegible text by using contextual priors and pixel‑level alignment. Hawkeye performs comparably to larger VLMs overall and surpasses them on text at or below 20 pixels, making it suitable for real‑time deployment in noisy egocentric settings.

By Brian Tang, Kang G. Shin