AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

11,407 stories · RSS feed

arXiv AI

Jun 4

FinTradeBench: A Financial Reasoning Benchmark for LLMs

arXiv:2603. 19225v3 Announce Type: replace-cross Abstract: Real-world financial decision-making is a challenging problem that requires reasoning over heterogeneous signals, including company fundamentals derived from regulatory filings and trading signals computed from price dynamics.

By Yogesh Agrawal, Aniruddha Dutta, Md Mahadi Hasan, Santu Karmaker, Aritra Dutta

llms rag benchmarks safety

arXiv Machine Learning

Jun 4

Edge of Stability Selectively Shapes Learning Across the Data Distribution

arXiv:2606. 04212v1 Announce Type: new Abstract: Existing analyses of the edge of stability (EoS) treat it as a global property of optimization.

By Shauna Kwag, Anakha Ganesh, Tomaso Poggio, Pierfrancesco Beneventano

safety

arXiv AI

Jun 4

MorphoQuant: Modality-Aware Quantization for Omni-modal Large Language Models

arXiv:2606. 04349v1 Announce Type: cross Abstract: Conventional Post-Training Quantization (PTQ) methods struggle with 4-bit Omni-modal Large Language Models (OLLMs) due to the extreme distribution heterogeneity and disparate outlier patterns across modalities.

By Yue Wu, Changyuan Wang, Zixuan Wang, Shilin Ma, Yansong Tang

llms efficiency multimodal benchmarks safety

arXiv Machine Learning

Jun 4

Fixed Aggregation Features Can Rival GNNs

arXiv:2601. 19449v2 Announce Type: replace Abstract: Graph neural networks (GNNs) are widely believed to excel at node representation learning through trainable neighborhood aggregations.

By Celia Rubio-Madrigal, Rebekka Burkholz

llms benchmarks safety

arXiv Machine Learning

Jun 4

Vision Transformer Finetuning Benefits from Non-Smooth Components

arXiv:2602. 06883v3 Announce Type: replace Abstract: The smoothness of the transformer architecture has been extensively studied in the context of generalization, training stability, and adversarial robustness.

By Ambroise Odonnat, Laetitia Chapel, Romain Tavenard, Ievgen Redko

llms computer-vision fine-tuning safety

arXiv AI

Jun 4

From Agent Traces to Trust: Evidence Tracing and Execution Provenance in LLM Agents

arXiv:2606. 04990v1 Announce Type: cross Abstract: Large language model (LLM)-based agents increasingly solve complex tasks by interacting with external tools, retrieval systems, memory modules, environments, and other agents.

By Yiqi Wang, Jiaqi Zhang, Taotao Cai, Zirui Liu, Qingqiang Sun, Zequn Sun, Zhangkai Wu, Mingkai Zhang, Yanming Zhu

llms agents benchmarks safety

arXiv AI

Jun 4

Measuring What Matters: Synthetic Benchmarks for Concept Bottleneck Models

arXiv:2606. 04326v1 Announce Type: cross Abstract: Concept bottleneck models predict outcomes from high-level concepts detected in inputs.

By Julian Skirzynski, Harry Cheon, Shreyas Kadekodi, Meredith Stewart, Berk Ustun

benchmarks safety

arXiv AI

Jun 4

SCI-PRM: A Tool Aware Process Reward Model for Scientific Reasoning Verification

arXiv:2606. 04579v1 Announce Type: new Abstract: While Process Reward Models (PRMs) have achieved remarkable success in mathematical reasoning, their application in complex scientific domains-such as biology, chemistry, and physics remains largely unexplored.

By Xiangyu Zhao, Hengyuan Zhao, Yiheng Wang, Wanghan Xu, Yuhao Zhou, Qinglong Cao, Zhiwang Zhou, Lei Bai, Wenlong Zhang, Xiao-Ming Wu

reinforcement-learning safety

arXiv Machine Learning

Jun 4

Preserving Data Privacy in Learning Causal Structure with Fully Homomorphic Encryption

arXiv:2606. 05129v1 Announce Type: cross Abstract: Preserving data privacy is an important topic in structural data management and data mining.

By Jian Yang, Yuan Tong, Qinbin Li, Zeyi Wen, Xiaofang Zhou

safety

arXiv Machine Learning

Jun 4

Inverse Critical Experiment Design via Gradient Optimization and a Multigroup Attention-Based Neural Network Architecture

arXiv:2606. 04033v1 Announce Type: new Abstract: The validation of advanced nuclear reactor designs and fuel concepts requires critical experiments with high neutronic similarity to the target technology.

By Will Savage, Logan Burnett, Dean Price

safety

arXiv Machine Learning

Jun 4

Federated Learning for Multi-Center Sepsis Early Prediction with Privacy-Preserving

arXiv:2606. 04338v1 Announce Type: new Abstract: Privacy-sensitive and distributed characteristics of multi-center medical data bring severe obstacles to centralized modeling for accurate early prediction of sepsis.

By Xixi Tian, Di Wu, Xiang Liu, Yiziting Zhu, Yujie Li, Xin Shu, Bin Yi

safety

arXiv AI

Jun 4

Why Muon Outperforms Adam: A Curvature Perspective

arXiv:2606. 04662v1 Announce Type: cross Abstract: Muon improves training efficiency over Adam in large language-model training by about two times, but the local geometric source of this advantage remains unclear.

By Shuche Wang, Fengzhuo Zhang, Jiaxiang Li, Dirk Bergemann, Zhuoran Yang

safety

arXiv AI

Jun 4

Quantum entanglement provides a competitive advantage in adversarial games

arXiv:2603. 10289v2 Announce Type: replace-cross Abstract: Whether uniquely quantum resources confer advantages in fully classical, competitive environments remains an open question.

By Peiyong Wang, Kieran Hymas, James Quach

agents reinforcement-learning safety

arXiv Machine Learning

Jun 4

Variance Reduction for Heavy-Tailed Monetization Metrics in Ranking Experiments via Post-Stratification

arXiv:2606. 04110v1 Announce Type: new Abstract: Online evaluation of ranking and retrieval systems often relies on downstream monetization metrics such as app revenue or creator earnings.

By Neeti Pokharna, Olivier Jeunen, Yatharth Saraf, Aleksei Ustimenko

safety

arXiv Machine Learning

Jun 4

Towards Pretraining Text Encoders for TabPFN

arXiv:2606. 04876v1 Announce Type: new Abstract: Tabular foundation models, such as TabPFN, achieve strong performance on tabular datasets with numerical and categorical data, but do not natively handle high-cardinality text features.

By Mustafa Tajjar, Alexander Pfefferle, Lennart Purucker, Frank Hutter

llms rag safety

arXiv AI

Jun 4

Semiparametric Preference Optimization: Your Language Model is Secretly a Single-Index Model

arXiv:2512. 21917v3 Announce Type: replace-cross Abstract: Policy alignment to preference data typically assumes a known link function between observed preferences and latent rewards (e.

By Nathan Kallus

llms safety

arXiv Machine Learning

Jun 4

NLLog: Lightweight, Explainable SOC Anomaly Detection via Log-to-Language Rewriting

arXiv:2606. 04957v1 Announce Type: cross Abstract: System-generated logs underpin security monitoring, yet their rigid template-based format hinders both automated analysis and human comprehension.

By Samuel Ndichu, Tao Ban, Seiichi Ozawa, Takeshi Takahashi, Daisuke Inoue

safety

arXiv AI

Jun 4

MM-BizRAG: Rethinking Multimodal Retrieval-Augmented Generation for General Purpose Enterprise Q&A

arXiv:2606. 04231v1 Announce Type: cross Abstract: Recent advances in multimodal retrieval-augmented generation (MM-RAG) have shifted toward minimal parsing, relying on page-level images for producing retriever embeddings and for answer generation.

By Hanoz Bhathena, Parin Rajesh Jhaveri, Rohan Mittal, Prateek Singh, Aymen Kallala, Rachneet Kaur, Yiqiao Jin, Zhen Zeng, Adwait Ratnaparkhi, Denis Kochedykov

llms rag fine-tuning multimodal benchmarks safety

arXiv AI

Jun 4

Trace-Mediated Peak Bias: Bridging Temporal Credit Assignment and Cognitive Heuristics in Deep Reinforcement Learning

arXiv:2606. 04735v1 Announce Type: cross Abstract: Temporal credit assignment is central to both biological and artificial intelligence, yet its interaction with non-linear function approximation is poorly understood.

By Viktor Vesel\'y, Aleksandar Todorov, Erwan Escudie, Matthia Sabatelli

agents reinforcement-learning safety

arXiv AI

Jun 4

PerceptTwin: Semantic Scene Reconstruction for Iterative LLM Planning and Verification

arXiv:2606. 04226v1 Announce Type: cross Abstract: Simulation environments are useful for both robot policy learning and planning verification and validation.

By Charlie Gauthier, Sacha Morin, Liam Paull

llms robotics safety

Get the daily briefing