AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,798 stories · RSS feed

arXiv Machine Learning

Oct 7

Towards Explainable Benchmarking for Data-driven Post-Wildfire Debris Flow Prediction

arXiv:2610.07358v1 Announce Type: new Abstract: Post-wildfire debris flows (PFDFs) are destructive sediment-laden hazards triggered when intense rainfall strikes recently burned terrain, destabilizin...

By Zhisheng Qi, Li Zhu, Utkarsh Sahu, Douglas Tommey, Josh Roering, Yu Wang

reinforcement-learning benchmarks safety

arXiv Machine Learning

Oct 7

Adversarial Training for Deep Hedging in Nonstationary Markets

arXiv:2610.07162v1 Announce Type: new Abstract: Deep hedging learns trading policies from historical or simulated market trajectories, yet under nonstationarity these training paths may not represent...

By Philipp J. Schneider, Lukas Looser, Antoine Garin, Shuhan Liu, Daniel Kuhn

diffusion safety

arXiv AI

Oct 7

Zero-Shot Visualization: Exploring Text Corpora with User-Prompted Axes

arXiv:2610.06889v1 Announce Type: cross Abstract: We study the application of large language models (LLMs) to the visual exploration of textual corpora. We introduce zero-shot visualization (ZSV), a...

By Arnau Bueno Tricas, Jose A. Rodr\'iguez-Serrano

llms rag benchmarks safety

arXiv AI

Oct 7

Modeling Latent Disturbances for Robust Decision-Making in World Models

arXiv:2610.07599v1 Announce Type: cross Abstract: In this paper, we study robust decision-making in the latent space of world models (WMs). Robust optimization is a mathematical framework where, give...

By Junwon Seo, Andrea Bajcsy

diffusion robotics safety

arXiv Computation and Language

Oct 7

Model in Distress: Sentiment Analysis on French Synthetic Social Media

arXiv:2604.18226v2 Announce Type: replace Abstract: Automated analysis of customer feedback on social media is hindered by three challenges: the high cost of annotated training data, the scarcity of...

By Pierre-Carl Langlais, Pavel Chizhov, Yannick Detrois, Carlos Rosas-Hinostroza, Ivan P. Yamshchikov, Bastien Perroy

llms nlp fine-tuning benchmarks safety

arXiv Machine Learning

Oct 7

MS-ECG-FM: Towards a More Universal Electrocardiogram Foundation Model for Health Monitoring using Multi-source Contrastive Learning

arXiv:2610.07662v1 Announce Type: new Abstract: Electrocardiography (ECG) records the electrical activity of the heart, aiding diagnosis by detecting abnormalities in cardiac function. ECG foundation...

By Robert A. Lewis, I-Min Chiu, Kyle Verrier, Karthik Jayaraman Raghuram, Francoise Marvel, Salar Abbaspourazad, Anshuman Mishra, Guillermo Sapiro, Andrew C. Miller, Joseph Futoma

benchmarks safety

arXiv Machine Learning

Oct 7

TRACE: Rollout-Guided Quantization-Aware Training for FP4 Reinforcement Learning of MoE Language Models

arXiv:2610.07767v1 Announce Type: new Abstract: Reinforcement learning (RL) for post-training large language models (LLMs) incurs substantial computation and memory overhead during rollout generation...

By Xin Wang, Hao Yu, Zhengyang Zhuge, Bochao Mao, Zheng Li, Junda Feng, Yuyan Luo, Yi Zhang, Yizhong Cao, Mi Zhang, Dayiheng Liu, Jianwei Zhang

llms reinforcement-learning efficiency safety

arXiv Machine Learning

Oct 7

ApexQuant: Data-Free Elastic Quantization by Residual Re-Isotropization

arXiv:2610.07904v1 Announce Type: new Abstract: We introduce ApexQuant, a calibration-free quantization method that recursively re-quantizes the residual error, serving as a refinement layer on top o...

By Aksel Fristrup, Sumit Pandey, Ankit Kariryaa

llms efficiency safety

arXiv Machine Learning

Oct 7

Optimization Encoders: Rethinking Second-Order Meta-Learning for Neural Fields

arXiv:2610.08075v1 Announce Type: new Abstract: Conditional neural fields represent signals continuously, but their effectiveness depends on how the conditional latent representations are inferred fr...

By Rudolf L. M. van Herten, Soufiane Ben Haddou, Rachit Saluja, Johannes C. Paetzold

llms computer-vision safety

arXiv Machine Learning

Oct 7

Data Fusion for Errors-in-Variables

arXiv:2610.07048v1 Announce Type: cross Abstract: We study errors-in-variables problems in which a target study contains only a single error-prone surrogate of an unobserved exposure, while an extern...

By Huali Zhao, Molei Liu, Tianying Wang

safety

arXiv Machine Learning

Oct 7

Interleaved Projected Gradient Descent for Safe Imitation Learning

arXiv:2610.07167v1 Announce Type: cross Abstract: We propose an imitation-learning design for neural-network control policies under state and input constraints. Training alternates a standard imitati...

By Shengfan Cao, Francesco Borrelli

agents safety

arXiv Machine Learning

Oct 7

An overview of machine learning-enhanced iterative methods for systems of linear and nonlinear equations

arXiv:2610.07211v1 Announce Type: cross Abstract: Systems of equations arise in a wide range of scientific and engineering applications. The present work focuses on solvers for general systems of equ...

By Yuhuang Meng, Jing Zhao, Alexander Heinlein

benchmarks safety

arXiv Machine Learning

Oct 7

On Color Alignment in VAE Latent Spaces and Its Applications

arXiv:2610.07072v1 Announce Type: cross Abstract: Variational autoencoders (VAEs) are a key part of modern text-to-image models, which generate images within their latent space. VAEs are known to dis...

By Julian D. Santamaria, Kai Wang, Jes\'us Malo, Javier Vazquez-Corral, Alexandra G\'omez-Villa

diffusion benchmarks safety

arXiv Machine Learning

Oct 7

The Model Plants the Trigger: Answer-Side Backdoor Attacks in Multi-Turn Large Language Models

arXiv:2610.07723v1 Announce Type: cross Abstract: Safety alignment in Large Language Models (LLMs) remains vulnerable to backdoor attacks. Existing LLM backdoors are almost all input-centric: activat...

By Yibo Zhang, Tianrong Guan, Liang Lin, Puze Wang, Jin Wang, Qingsong Wen

llms safety

arXiv Machine Learning

Oct 7

Rethinking Faithfulness in LLMs: A Pairwise Context-Sensitive Perspective

arXiv:2610.07894v1 Announce Type: cross Abstract: Large language models (LLMs) are expected to answer questions faithfully based on the provided context, abstaining when the context information is in...

By Zizhuo Zhang, Xiong Peng, Jingwei Sun, Rong Yao, Borui Jiang, Bo Han

llms benchmarks safety

arXiv Machine Learning

Oct 7

MSPR: Multi-scale Predictive Representations for Goal-conditioned Reinforcement Learning

arXiv:2605.09364v2 Announce Type: replace Abstract: This paper investigates robust representation learning in offline goal-conditioned reinforcement learning (GCRL). Particularly in sparse reward sce...

By Valliappan Chidambaram Adaikkappan, Sai Rajeswar, Pietro Mazzaglia, David Meger

diffusion reinforcement-learning benchmarks safety

arXiv Machine Learning

Oct 7

ZonoGPT: Towards An Abstract Domain for Verifying Large GPT Models

arXiv:2609.34457v2 Announce Type: replace Abstract: Transformer-based models are widely used for reasoning, coding, and multimodal agentic tasks. To provide formal assurance of desirable behaviors, s...

By Hai Duong, Thanh Le, ThanhVu Nguyen

llms agents multimodal safety

arXiv Computation and Language

Oct 7

Detecting LLM-Assisted Vietnamese Writing via Keystrokes under Behavioral Manipulation

arXiv:2610.07700v1 Announce Type: new Abstract: We study the robustness of keystroke dynamics for detecting large language model (LLM)-assisted writing. We introduce a Vietnamese keystroke dataset ca...

By Thanh Dong, An Ngo, Minh Dau, Rajesh Kumar

llms robotics safety

arXiv Computation and Language

Oct 7

Reading, Not Manipulating: Leveraging Router Logits for Multimodal Safety in MoE Vision-Language Models

arXiv:2610.07774v1 Announce Type: new Abstract: Vision-language models (VLMs) face compositional safety risks where harmful intent emerges from the interaction between visual and textual inputs. As m...

By Ziyuan Yang, Wenxuan Ding, Shangbin Feng, Yulia Tsvetkov

llms fine-tuning multimodal benchmarks safety

arXiv Computation and Language

Oct 7

OMIT the Action: Measuring Framing-Invariant Omission Bias under Philosophical Disagreement

arXiv:2610.07847v1 Announce Type: new Abstract: As LLMs increasingly assist in moral reasoning, omission bias, the tendency to prefer inaction even when equivalent framings reverse substantive outcom...

By Sihyeon Lee, Jihun Song, Chanwoo Kim, Jiwoo Kum, Chanjun Park

llms benchmarks safety

Get the daily briefing