AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

11,365 stories · RSS feed

arXiv Machine Learning

Jun 5

Conformal Risk-Averse Decision Making with Action Conditional Guarantee

arXiv:2606. 05551v1 Announce Type: cross Abstract: Reliable decision making pipelines powered by machine learning models require uncertainty quantification (UQ) methods that come with explicit safety guarantees.

By Zihan Zhu, Shayan Kiyani, George Pappas. Hamed Hassani

safety

arXiv Machine Learning

Jun 5

Bridging Domain Expertise and Generalization for Performance Estimation

arXiv:2606. 06335v1 Announce Type: new Abstract: Performance estimation under distribution shift aims to predict how a model behaves on an unlabeled test set whose distribution differs from the training data, a scenario that requires reliable indicators that can faithfully reflect model behavior without ground-truth labels.

By Shuxuan Li, Zhilin Zhao, Quyu Kong, Wei-Shi Zheng

safety

arXiv Machine Learning

Jun 5

Rotation-Parameterized Graph Fractional Fourier Transform: Definition, Properties, and Optimal Filtering

arXiv:2511. 16111v2 Announce Type: replace-cross Abstract: Graph spectral representations are fundamental in graph signal processing, providing a rigorous frameworkforanalyzing graph-structured data.

By Feiyue Zhao, Mingzhi Wang, Yangfan He, Zhichao Zhang

safety

arXiv Machine Learning

Jun 5

HANDOFF: Humanoid Agentic Task-Space Whole-Body Control via Distilled Complementary Teachers

arXiv:2606. 06493v1 Announce Type: cross Abstract: For a humanoid robot to be deployed in the real world, the choice of command space (i.

By Lizhi Yang, Junheng Li, Nehar Poddar, Yiling Hou, Gio Huh, Robert Griffin, Georgia Gkioxari, Aaron Ames

agents robotics fine-tuning efficiency multimodal benchmarks safety

arXiv Machine Learning

Jun 5

Towards AI epidemiology: a measurement standardisation framework for prospective risk detection

arXiv:2512. 15783v3 Announce Type: replace-cross Abstract: This paper proposes a measurement standardisation framework that compresses expert-AI interactions into structured, comparable fields for prospective risk detection in deployed AI systems, without access to model internals.

By Kit Tempest-Walters

llms safety

arXiv Machine Learning

Jun 5

LEVANTE-bench: Multi-Scale Comparison of VLMs to Children Using Cognitive Tasks (or, "Is Your VLM Smarter Than a 5th Grader?")

arXiv:2606. 05497v1 Announce Type: new Abstract: Given the inherently multimodal nature of human experience, vision-language models (VLMs) hold substantial promise for modeling human cognition as it grows and develops with experience.

By Alvin Wei Ming Tan, David Cardinal, Tania Lorido-Botran, Laura Bravo-Sanchez, Sunny Yu, Michael C. Frank

llms multimodal benchmarks safety

arXiv Machine Learning

Jun 5

Let It Be Simple: One-Step Action Generation for Vision-Language-Action Models

arXiv:2606. 05737v1 Announce Type: cross Abstract: Diffusion-based vision-language-action (VLA) models often inherit the image-generation view: actions are generated by iterative denoising.

By Yitong Chen, Shiduo Zhang, Jingjing Gong, Xipeng Qiu

diffusion robotics efficiency multimodal safety

arXiv Machine Learning

Jun 5

Grounded but Misleading: Evaluating Semantic Alignment in AI-Generated Security Explanations

arXiv:2602. 05056v2 Announce Type: replace-cross Abstract: Online scams increasingly leverage fluent and context-aware social engineering strategies, creating growing demand for AI systems that explain why a message may be risky.

By Heajun An, Connor Ng, Sandesh Sharma Dulal, Junghwan Kim, Jin-Hee Cho

llms safety

arXiv Machine Learning

Jun 5

Concept-SAE: A Controllable and Invertible Concept Interface for Sparse Autoencoders

arXiv:2509. 22015v2 Announce Type: replace Abstract: Standard Sparse Autoencoders (SAEs) excel at discovering a dictionary of a model's learned features, providing a powerful lens for passive feature discovery.

By Jianrong Ding, Muxi Chen, Chenchen Zhao, Qiang Xu

safety

arXiv Machine Learning

Jun 5

Adaptive Oscillatory-State Alignment for Time Series Forecasting

arXiv:2606. 06010v1 Announce Type: new Abstract: Long-term time series forecasting benefits from inductive biases that expose recurring temporal structure.

By Zhangyao Song, Ziqiong Li, Xiangfei Qiu, Chao Zha, Yinfei Xu, Tao Guo

efficiency benchmarks safety

arXiv Machine Learning

Jun 5

Do MLLMs Capture How Interfaces Guide User Behavior? A Benchmark for Multimodal UI/UX Design Understanding

arXiv:2505. 05026v5 Announce Type: replace-cross Abstract: User interface (UI) design goes beyond visuals to shape user experience (UX), underscoring the shift toward UI/UX as a unified concept.

By Jaehyun Jeon, Min Soo Kim, Jang Han Yoon, Sumin Shim, Yejin Choi, Hanbin Kim, Dae Hyun Kim, Youngjae Yu

llms multimodal benchmarks safety

arXiv Machine Learning

Jun 5

Temporal Preference Concepts and their Functions in a Large Language Model

arXiv:2606. 05194v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly being deployed to make decisions that require trading off near-term gains against long-term consequences, yet little is known about how they internally represent or resolve these tradeoffs.

By Ian Rios-Sialer, Shantanu Darveshi, Shuai Jiang, Avigya Paudel, Anastasiia Pronina, Ipshita Bandyopadhyay, Justin Shenk

llms safety

arXiv Machine Learning

Jun 5

Can LLMs Write Correct TLA+ Specifications? Evaluating Natural-Language-to-TLA+ Generation

arXiv:2606. 05792v1 Announce Type: cross Abstract: TLA+ has supported industrial verification at companies such as Amazon and Microsoft, yet writing correct TLA+ specifications from natural language still requires time and expertise, which limits adoption.

By Arslan Bisharat, Brian Ortiz, Eric Spencer, Khushboo Bhadauria, TaiNing Wang, George K. Thiruvathukal, Konstantin Laufer, Mohammed Abuhamad

llms safety

arXiv Machine Learning

Jun 5

Two-Way Is Better Than One: Bidirectional Alignment with Cycle Consistency for Exemplar-Free Class-Incremental Learning

arXiv:2606. 05675v1 Announce Type: new Abstract: Continual learning (CL) seeks models that acquire new skills without erasing prior knowledge.

By Hongye Xu, Bartosz Krawczyk

rag benchmarks safety

arXiv Machine Learning

Jun 5

Quantifying the Privacy of Counterfactuals by Leveraging Membership Inference Attacks Against Synthetic Data

arXiv:2606. 06334v1 Announce Type: new Abstract: Counterfactuals are typically used in high-stakes decision areas to explain a machine learning model by showing how changes to the user profiles result in the desired outcome.

By Maryam Babaei, Yingke Wang, Hadrien Lautraite, Heber H. Arcolezi, Ulrich Aivodji, Sebastien Gambs

safety

arXiv Machine Learning

Jun 5

Subspace-Aware Sparse Autoencoders for Effective Mechanistic Interpretability

arXiv:2606. 06333v1 Announce Type: new Abstract: Sparse Autoencoders (SAEs) are widely used for mechanistic interpretability in large language models, yet their formulation assigns each latent feature a single decoder direction, implicitly assuming features to be one-dimensional.

By Seyed Arshan Dalili, Mehrdad Mahdavi

llms efficiency safety

arXiv Machine Learning

Jun 5

TLA-Prover: Verifiable TLA+ Specification Synthesis via Preference-Optimized Low-Rank Adaptation

arXiv:2606. 06133v1 Announce Type: cross Abstract: TLA+ is a formal specification language for verifying distributed systems and safety-critical protocols.

By Eric Spencer, Arslan Bisharat, Brian Ortiz, Khushboo Bhadauria, TaiNing Wang, George K. Thiruvathukal, Konstantin Laufer, Mohammed Abuhamad

llms reinforcement-learning fine-tuning benchmarks safety

arXiv Machine Learning

Jun 5

Domain-Adapted Small Language Models with Hybrid Post-Processing: Achieving Cost-Efficient, Low-Latency Multi-Label Structured Prediction via LoRA Fine-Tuning on Scarce Data

arXiv:2606. 05781v1 Announce Type: new Abstract: Deploying frontier large language models (LLMs) for domain-specific structured evaluation tasks often incurs substantial latency, cost, and data privacy overhead.

By Srinivasan Manoharan, Dilipkumar Nallusamy, Sachin Kumar, Haifeng Wu

llms fine-tuning safety

arXiv Machine Learning

Jun 5

RedditPersona: A Modular Framework for Community-Conditioned LLM Adaptation from Reddit

arXiv:2606. 06027v1 Announce Type: cross Abstract: Community-conditioned language model adaptation requires choices about data collection, community definition, and evaluation that are currently made independently in each study, making it hard to compare assumptions or reuse artifacts.

By Amirhossein Ghaffari, Ali Goodarzi, Huong Nguyen, Simo Hosio, Lauri Lov\'en, Ekaterina Gilman

llms fine-tuning safety

arXiv Machine Learning

Jun 5

Statistically Reliable LLM-Based Ranking Evaluation via Prediction-Powered Inference

arXiv:2606. 05308v1 Announce Type: new Abstract: With PRECISE, we extended Prediction-Powered Inference to produce bias-corrected estimates of ranking evaluation metrics by combining a small human-labeled set with a large LLM-judged set.

By Abhishek Divekar

llms benchmarks safety

Get the daily briefing