AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

11,407 stories · RSS feed

arXiv Machine Learning

Jun 5

CASS-RTL: Correctness-Aware Subspace Steering for RTL Generation with LLMs

arXiv:2606. 05680v1 Announce Type: cross Abstract: Recent advances in large language models (LLMs) have enabled the automatic synthesis (generation) of register-transfer level (RTL) code from natural language instructions, offering a promising pathway to accelerate chip design.

By Mohammad Akyash, Nowfel Mashnoor, Kimia Azar, Hadi Kamali

llms rag agents fine-tuning safety

arXiv Machine Learning

Jun 5

Selective-Advantage Entropy-Adaptive Horizon GRPO: Asymmetric Token-Level Discounting for Efficient Reinforcement Learning of Language Models

arXiv:2606. 05434v1 Announce Type: new Abstract: Group Relative Policy Optimisation (GRPO) has emerged as an effective reinforcement-learning algorithm for aligning language models on reasoning tasks, but it treats every token position and every sampled rollout symmetrically.

By Chirag Chawla, Rohan Charudatt Salvi, Madhav S. Baidya

llms reinforcement-learning fine-tuning benchmarks safety

arXiv Machine Learning

Jun 5

3D Underwater Path Planning via Generative Flow Field Surrogates

arXiv:2606. 06077v1 Announce Type: cross Abstract: Autonomous underwater vehicle (AUV) launch and recovery (LAR) into the hull of an advancing host platform requires traversal of a complex, three-dimensional propeller wake whose hydrodynamic structure cannot be characterised by a uniform current model.

By Zachary Cooper-Baldock, Paulo E. Santos, Russell S. A. Brinkworth, Karl Sammut

agents robotics benchmarks safety

arXiv Machine Learning

Jun 5

Path-Coupled Bellman Flows for Distributional Reinforcement Learning

arXiv:2605. 08253v2 Announce Type: replace Abstract: Distributional reinforcement learning (DRL) models the full return distribution, but existing finite-support or quantile-based methods rely on projections, while recent flow-based approaches can suffer from \emph{boundary mismatch} at the flow source or from \emph{high-variance} bootstrapping when current and successor noises are independent.

By Boyang Xu, Qing Zou, Siqin Yang, Hao Yan

diffusion reinforcement-learning safety

arXiv Machine Learning

Jun 5

SHALA-LLM: Smartly Handling Ambiguous Labels in Aligning LLMs

arXiv:2606. 05376v1 Announce Type: new Abstract: Many human-centered tasks, including natural language inference (NLI) and emotion recognition (ER), have multiple plausible interpretations, leading to label ambiguity and challenging disagreements across human annotators.

By Jingyao Wu, Ashley Wang, Keane Ong, Paul Pu Liang, Rosalind Picard

llms reinforcement-learning benchmarks safety

arXiv Machine Learning

Jun 5

Generalized TV--$\ell_p$ Structured Priors for Bayesian $T_1$ Mapping

arXiv:2606. 05381v1 Announce Type: new Abstract: We propose an extended family of structured spatial priors that incorporates the total variation (TV) function with $\ell_p$ norms.

By Disi Lin, Martin Berggren, Tommy L\"ofstedt

rag safety

arXiv Machine Learning

Jun 5

LLMs for Secure Hardware Design and Related Problems: Opportunities and Challenges

arXiv:2605. 10807v4 Announce Type: replace-cross Abstract: The integration of Large Language Models (LLMs) into Electronic Design Automation (EDA) and hardware security is rapidly reshaping the semiconductor industry.

By Johann Knechtel, Ozgur Sinanoglu, Ramesh Karri

llms agents benchmarks safety

arXiv Machine Learning

Jun 5

Conformal Risk-Averse Decision Making with Action Conditional Guarantee

arXiv:2606. 05551v1 Announce Type: cross Abstract: Reliable decision making pipelines powered by machine learning models require uncertainty quantification (UQ) methods that come with explicit safety guarantees.

By Zihan Zhu, Shayan Kiyani, George Pappas. Hamed Hassani

safety

arXiv Machine Learning

Jun 5

HANDOFF: Humanoid Agentic Task-Space Whole-Body Control via Distilled Complementary Teachers

arXiv:2606. 06493v1 Announce Type: cross Abstract: For a humanoid robot to be deployed in the real world, the choice of command space (i.

By Lizhi Yang, Junheng Li, Nehar Poddar, Yiling Hou, Gio Huh, Robert Griffin, Georgia Gkioxari, Aaron Ames

agents robotics fine-tuning efficiency multimodal benchmarks safety

arXiv Machine Learning

Jun 5

Quantifying the Privacy of Counterfactuals by Leveraging Membership Inference Attacks Against Synthetic Data

arXiv:2606. 06334v1 Announce Type: new Abstract: Counterfactuals are typically used in high-stakes decision areas to explain a machine learning model by showing how changes to the user profiles result in the desired outcome.

By Maryam Babaei, Yingke Wang, Hadrien Lautraite, Heber H. Arcolezi, Ulrich Aivodji, Sebastien Gambs

safety

arXiv Machine Learning

Jun 5

Soft Sequence Policy Optimization

arXiv:2602. 19327v3 Announce Type: replace Abstract: A significant portion of recent research on Large Language Model (LLM) alignment focuses on developing new policy optimization methods based on Group Relative Policy Optimization (GRPO).

By Svetlana Glazyrina, Maksim Kryzhanovskiy, Roman Ischenko

llms reinforcement-learning safety

arXiv Machine Learning

Jun 5

Can LLMs Write Correct TLA+ Specifications? Evaluating Natural-Language-to-TLA+ Generation

arXiv:2606. 05792v1 Announce Type: cross Abstract: TLA+ has supported industrial verification at companies such as Amazon and Microsoft, yet writing correct TLA+ specifications from natural language still requires time and expertise, which limits adoption.

By Arslan Bisharat, Brian Ortiz, Eric Spencer, Khushboo Bhadauria, TaiNing Wang, George K. Thiruvathukal, Konstantin Laufer, Mohammed Abuhamad

llms safety

arXiv Machine Learning

Jun 5

RedditPersona: A Modular Framework for Community-Conditioned LLM Adaptation from Reddit

arXiv:2606. 06027v1 Announce Type: cross Abstract: Community-conditioned language model adaptation requires choices about data collection, community definition, and evaluation that are currently made independently in each study, making it hard to compare assumptions or reuse artifacts.

By Amirhossein Ghaffari, Ali Goodarzi, Huong Nguyen, Simo Hosio, Lauri Lov\'en, Ekaterina Gilman

llms fine-tuning safety

arXiv Machine Learning

Jun 5

Adaptive Oscillatory-State Alignment for Time Series Forecasting

arXiv:2606. 06010v1 Announce Type: new Abstract: Long-term time series forecasting benefits from inductive biases that expose recurring temporal structure.

By Zhangyao Song, Ziqiong Li, Xiangfei Qiu, Chao Zha, Yinfei Xu, Tao Guo

efficiency benchmarks safety

arXiv Machine Learning

Jun 5

LEVANTE-bench: Multi-Scale Comparison of VLMs to Children Using Cognitive Tasks (or, "Is Your VLM Smarter Than a 5th Grader?")

arXiv:2606. 05497v1 Announce Type: new Abstract: Given the inherently multimodal nature of human experience, vision-language models (VLMs) hold substantial promise for modeling human cognition as it grows and develops with experience.

By Alvin Wei Ming Tan, David Cardinal, Tania Lorido-Botran, Laura Bravo-Sanchez, Sunny Yu, Michael C. Frank

llms multimodal benchmarks safety

arXiv Machine Learning

Jun 5

Escaping the Verifier: Learning to Reason via Demonstrations

arXiv:2511. 21667v4 Announce Type: replace Abstract: Training Large Language Models (LLMs) to reason often relies on Reinforcement Learning (RL) with task-specific verifiers.

By Locke Cai, Max Ryabinin, Ivan Provilkov

llms reinforcement-learning safety

arXiv Machine Learning

Jun 5

Beyond Output Matching: Preserving Internal Geometry in NVFP4 LLM Distillatio

arXiv:2606. 05682v1 Announce Type: cross Abstract: Demand for low-precision inference, including NVFP4-based approaches, has grown as large language models are increasingly deployed in latency and cost constrained production environments.

By Fangbo Tu, Junhua Zhao, Chi Liu, Xin Chen, Haifeng Wu, Jian Wan, Srinivasan Manoharan

llms reinforcement-learning efficiency safety

arXiv Machine Learning

Jun 5

FATE: Focal-modulated Attention Encoder for Multivariate Time-series Forecasting

arXiv:2408. 11336v3 Announce Type: replace Abstract: Climate change stands as one of the most pressing global challenges of the twenty-first century, with far-reaching consequences such as rising sea levels, melting glaciers, and increasingly extreme weather patterns.

By Tajamul Ashraf, Janibul Bashir

llms benchmarks safety

arXiv Machine Learning

Jun 5

When Denser Credit Is Not Enough: Evidence-Calibrated Policy Optimization for Long-Horizon LLM Agent Training

arXiv:2606. 05885v1 Announce Type: new Abstract: Long-horizon LLM agents require reinforcement learning methods that can assign credit to intermediate decisions under sparse and delayed rewards.

By Yuanfan Li, Qi Zhou, Wenjing Duan, Lu Chen

llms agents reinforcement-learning safety

arXiv Machine Learning

Jun 5

Symb-xMIL: Symbolic Explanations for Multiple Instance Learning in Digital Pathology

arXiv:2606. 06224v1 Announce Type: cross Abstract: Explanations of multiple instance learning (MIL) models are widely used for validation and discovery in digital histopathology.

By Yanqing Luo (Berlin Institute for the Foundations of Learning and Data, Berlin, Germany, Machine Learning Group, Technische Universit\"at Berlin, Berlin, Germany), Julius Hense (Berlin Institute for the Foundations of Learning and Data, Berlin, Germany, Machine Learning Group, Technische Universit\"at Berlin, Berlin, Germany), Niklas Preni{\ss}l (Institute of Pathology, Charit\'e Universit\"atsmedizin, Berlin, Germany, Berlin Institute of Health at Charit\'e -- Universit\"atsmedizin Berlin, BIH Biomedical Innovation Academy, BIH Charit\'e Digital Clinician Scientist Program, Berlin, Germany), Andreas Mock (Institute of Pathology, Ludwig Maximilian University of Munich, Munich, Germany, Division of Translational Medical Oncology, DKFZ, Heidelberg, Germany, NCT Heidelberg, Heidelberg, Germany, German Cancer Consortium), Klaus-Robert M\"uller (Berlin Institute for the Foundations of Learning and Data, Berlin, Germany, Machine Learning Group, Technische Universit\"at Berlin, Berlin, Germany, Department of Artificial Intelligence, Korea University, Seoul, Korea, Max-Planck Institute for Informatics, Saarbr\"ucken, Germany), Thomas Schnake (Department of Chemistry, Chemical Physics Theory Group, University of Toronto, Canada, Vector Institute for Artificial Intelligence, Toronto, Canada, Acceleration Consortium, University of Toronto, Canada), Mina Jamshidi Idaji (Berlin Institute for the Foundations of Learning and Data, Berlin, Germany, Machine Learning Group, Technische Universit\"at Berlin, Berlin, Germany)

safety

Get the daily briefing