AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

8,776 stories · RSS feed

arXiv Machine Learning
Jul 30

Scores Are Not Decisions: Cost-Aware Stopping for Tool Acquisition in LLM Agents

arXiv:2607. 27083v1 Announce Type: new Abstract: As LLM agents increasingly depend on diverse external services such as search engines, databases, and connectors, agent harnesses face a fundamental tool-selection challenge: acquiring too few tools leaves the task under-informed, while too many adds cost, context load, and privacy exposure.

By Yicheng Feng, Yan Zhang, Yan Cheng, Wei Qi
arXiv Machine Learning
Jul 30

Challenges and proposed solutions in modeling multimodal medical data: A systematic review

arXiv:2505. 06945v5 Announce Type: replace Abstract: Multimodal data modeling has emerged as a powerful approach in clinical research, enabling the integration of diverse data types such as imaging, genomics, wearable sensors, and electronic health records.

By Maryam Farhadizadeh, Maria Weymann, Michael Bla{\ss}, Johann Kraus, Christopher Gundler, Sebastian Walter, Noah Hempen, Hannah Bast, Harald Binder, Nadine Binder
arXiv Machine Learning
Jul 30

GPT-Red: Automated Red Teaming via Self-Play at Scale

arXiv:2607. 26115v1 Announce Type: cross Abstract: We introduce \textbf{GPT-Red}, an automated red-teaming agent that is trained to discover novel prompt injection attacks against frontier LLMs.

By Eric Wallace, Christopher A. Choquette-Choo, Nikhil Kandpal, Sam Toyer, Dylan Hunn, Stephanie Lin, Yuxin Wen, Xiangyu Qi, Christopher Wolff, Zizhao Wang, Milad Nasr, Sicheng Zhu, Chuan Guo, Juan Felipe Cer\'on Uribe, Kaiwen Wang, Aiden Low, Kai Xiao, Kai Chen
arXiv Machine Learning
Jul 30

Interpretable GOHR Agents via Sparse Autoencoders

arXiv:2607. 25132v2 Announce Type: replace Abstract: A central challenge in interpreting learned decision-making systems is to determine whether their internal representations contain concepts that help explain their behavior.

By Shiwei Tan, Yusong Zhao, Weiyi Qin, Wentian Wang, Jacob Feldman, Lazaros K. Gallos, Paul B. Kantor, Vladimir Menkov, Hao Wang
arXiv Machine Learning
Jul 30

Data Fusion and Contrastive Alignment for Unconstrained IR Molecular Structure Elucidation

arXiv:2607. 26164v1 Announce Type: new Abstract: Automated molecular structure elucidation from infrared (IR) spectroscopy data has seen significant advancements in recent years, but its broad applicability is limited by a reliance on pre-determined chemical formulas provided as auxiliary model inputs.

By Ethan J. Mick, Campbell A. Sweet, Matthias J. Young, Derek T. Anderson
arXiv Machine Learning
Jul 30

Stable and Budget-Feasible Coalition Formation for Clustered Federated Learning: A Hedonic Potential-Game Approach

arXiv:2607. 26788v1 Announce Type: cross Abstract: Clustered federated learning benefits from organizing heterogeneous participants into coalitions that train coalition-specific models, but such clustering is sustainable only if participants prefer their assigned coalition and the required transfers are affordable.

By Cengis Hasan
arXiv Machine Learning
Jul 30

DP-MGTD: Privacy-Preserving Machine-Generated Text Detection via Adaptive Differentially Private Entity Sanitization

arXiv:2601. 04641v2 Announce Type: replace-cross Abstract: The deployment of Machine-Generated Text (MGT) detection systems necessitates processing sensitive user data, creating a fundamental conflict between authorship verification and privacy preservation.

By Lionel Z. Wang, Yusheng Zhao, Jiabin Luo, Xinfeng Li, Lixu Wang, Yinan Peng, Haoyang Li, XiaoFeng Wang, Wei Dong
arXiv Machine Learning
Jul 30

Parameterized Fair Resource Allocation under Diversity Constraints

arXiv:2607. 26485v1 Announce Type: cross Abstract: Resource allocation across multiple agent groups arises in many applications including e-commerce recommendation systems, housing assignment, and course allocation, and is commonly formulated as an optimization problem with diversity constraints to ensure group fairness.

By Keke Huang, Yik Yu Ng, Laks V. S. Lakshmanan, Xiaokui Xiao
arXiv Machine Learning
Jul 30

On-Policy Distillation for LLM Safety: A Routing Approach to Template-Robust Realignment

arXiv:2607. 27081v1 Announce Type: cross Abstract: Fine-tuning is the dominant paradigm for specializing large language models (LLMs), yet it exposes a critical vulnerability: malicious data providers can embed harmful behaviors into downstream corpora, creating models that retain professional skills while violating human values on demand.

By Yongjian Guo, Wanlun Ma, Lingyu Shen, Xi Xiao, Sheng Wen
arXiv Machine Learning
Jul 30

Cost-Sensitive Conformal Prediction and Human-in-the-Loop Abstention for Imbalanced High-Stakes Decision Support: A Multi-Domain Benchmark

arXiv:2607. 27143v1 Announce Type: new Abstract: High-stakes decision systems in credit scoring, fraud detection, healthcare, and industrial safety require reliable uncertainty quantification under severe class imbalance and asymmetric error costs.

By Manpreet Singh, Akshatha Srikantha, Shyamal Lakhanpal