AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,788 stories · RSS feed

arXiv AI

Oct 15

MILP the Gap: An Automated Reasoning Approach to Condorcet's Paradox

arXiv:2604.19851v2 Announce Type: replace-cross Abstract: Computational social choice studies how conflicting individual preferences are aggregated into a collective decision, with applications to el...

By Itai Zilberstein, Ruben Martins, Ratip Emin Berker

benchmarks safety

arXiv AI

Oct 15

PermaFrost-Attack: Stealth Pretraining Seeding(SPS) for planting Logic Landmines During LLM Training

arXiv:2604.22117v3 Announce Type: replace-cross Abstract: Aligned large language models (LLMs) remain vulnerable to adversarial manipulation, and their reliance on web-scale pretraining creates a sub...

By Harsh Kumar, Rahul Maity, Tanmay Joshi, Aman Chadha, Vinija Jain, Suranjana Trivedy, Amitava Das

llms robotics safety

arXiv AI

Oct 15

XL-SafetyBench: A Country-Grounded Cross-Cultural Benchmark for LLM Safety and Cultural Sensitivity

arXiv:2605.05662v2 Announce Type: replace-cross Abstract: Current LLM safety benchmarks are predominantly English-centric and often rely on translation, failing to capture country-specific harms. Mor...

By Dasol Choi, Eugenia Kim, Jaewon Noh, Sang Seo, Eunmi Kim, Myunggyo Oh, Yunjin Park, Brigitta Jesica Kartono, Josef Pichlmeier, Helena Berndt, Sai Krishna Mendu, Glenn Johannes Tungka, \"Ozlem G\"ok\c{c}e, Suresh Gehlot, Katherine Pratt, Amanda Minnich, Haon Park

llms multimodal benchmarks safety

arXiv AI

Oct 15

APEX: Assumption-free Projection-based Embedding eXamination Metric for Generative Distribution Evaluation

arXiv:2605.07786v4 Announce Type: replace-cross Abstract: As generative models achieve unprecedented visual quality, the gold standard for image evaluation remains traditional feature-distribution me...

By Caterina Gallegati, Monica Bianchini, Franco Scarselli, Vittorio Murino, Barbara Toniella Corradini

rag benchmarks safety

arXiv AI

Oct 15

LLM Judges Have Dark Current: Stratified Null and Utility Tests for LLM Judge Reliability

arXiv:2606.15610v2 Announce Type: replace-cross Abstract: Passing an exact-clone null gate does not entail passing a matched target-selection utility gate. We call a non-tie preference between exact...

By Hiroyasu Usami, Keisuke Hara, Ayato Tsuboi, Naohiko Matsuda

llms safety

arXiv AI

Oct 15

HALLMARK: How to Diagnose LLM Citation Verifiers and When to Deploy Them

arXiv:2607.18360v2 Announce Type: replace-cross Abstract: GPTZero found 53 NeurIPS 2025 accepted papers with hallucinated citations, and venues now screen submissions with automated reference checker...

By Kayoon Kim, Wieland Brendel, Patrik Reizinger

llms agents benchmarks safety

arXiv AI

Oct 15

From Portraits to Pathology: Probing Diagnostic Overreach in Vision-Language Models

arXiv:2609.05540v2 Announce Type: replace-cross Abstract: Many medical conditions require clinical assessment of symptoms, history, and functioning rather than judgments from appearance. We study whe...

By Karan Dua, Amit Agarwal, Hitesh Laxmichand Patel, Hansa Meghwani, Jyotika Singh, Ranjeet Gupta, Graham Horwood, Tao Sheng, Avi Sil, Sujith Ravi, Dan Roth

llms multimodal safety

arXiv AI

Oct 15

CrystalJev: calibrated foundation-model probabilities decide what to compute next in materials discovery

arXiv:2610.06985v2 Announce Type: replace-cross Abstract: Atomistic foundation models screen hypothetical crystals by thresholding predicted energies, an output that states neither how likely a candi...

By Peng Kang, Yaxuan Sun, Zhen Li, Yu Liu, Lei Zheng, Huibin Xu

safety

arXiv Machine Learning

Oct 15

Beyond Aggregate Accuracy: Item-Level Transitions in Prompt Optimization

arXiv:2610.19084v1 Announce Type: new Abstract: Automatic prompt optimization (APO) uses feedback from model responses to improve prompts automatically. However, its effectiveness is usually measured...

By Chinmoy Mitra, Tarek Mahmud

llms benchmarks safety

arXiv Machine Learning

Oct 15

FinGREAT: Evaluating LLMs on Financial Graph-guided Retrieval And Typing

arXiv:2610.19240v1 Announce Type: new Abstract: Financial disclosures express the underlying relationships of double-entry accounting through unstructured text. While financial LLM benchmarks predomi...

By Ashman Mehra, Bryan Routledge, Jane JaeYeon Pyo, Pierre Jinghong Liang

llms nlp benchmarks safety

arXiv Machine Learning

Oct 15

Did the LLM Answer the Question? A Semantic Reconstruction Game for Jailbreak Evaluation

arXiv:2610.19306v1 Announce Type: new Abstract: Jailbreak attacks against large language models (LLMs) are evaluated using attack success rate (ASR), which depends on correctly identifying responses...

By Liam Davis, Romain Emil-Popescu, Suman Jana

llms multimodal safety

arXiv Machine Learning

Oct 15

Comprehensive Comparisons of Reinforcement Learning Algorithms in Autonomous Driving

arXiv:2610.19429v1 Announce Type: new Abstract: This paper presents a comprehensive comparison of four representative reinforcement learning (RL) methods in autonomous vehicle (AV) decision-making, i...

By Dechen Xie, Mushuang Liu

agents reinforcement-learning robotics safety

arXiv Machine Learning

Oct 15

Bayesian Learning to Select Fair Classifiers

arXiv:2610.20514v1 Announce Type: new Abstract: Evaluations of classifiers (AI or human or agents) often assume a batch setting in which every agent produces labels on every example in a test set, wh...

By Saatvik Kher, Hyungrok Do, Judy Zhong, Padhraic Smyth

llms agents reinforcement-learning safety

arXiv Machine Learning

Oct 15

Asymptotically Exact Negative Guidance of Diffusion Models via Positive-Unlabeled Learning

arXiv:2610.19802v1 Announce Type: new Abstract: Negative guidance of diffusion models generates samples that avoid an undesired condition, specified either by reference samples or by a conditional sc...

By Hirohane Takagi, Masashi Sugiyama

diffusion safety

arXiv Machine Learning

Oct 15

MAEM: Mitigating Recency Bias via a Replay-Free, Capacity-Controlled Dynamic Mixture of Experts for Class-Incremental Graph Learning

arXiv:2610.20238v1 Announce Type: new Abstract: Graph Neural Networks (GNNs) excel on graph-structured data but suffer catastrophic forgetting when classes arrive sequentially. In class-incremental g...

By Junlin Chen, Wenhao Wang, Junchao Xu, Qingsong Yao, Fei Xue, Li Sun, Xiao Wang, Ruijie Wang

fine-tuning benchmarks safety

arXiv Machine Learning

Oct 15

From Spectral to Frobenius: When Muon Loses Its Spectral Implicit Bias

arXiv:2610.20112v1 Announce Type: new Abstract: Prior work on implicit bias shows that gradient descent selects the Frobenius max-margin direction on linearly separable problems, whereas Fan et al. (...

By Yongtao Wu, Mahdi Mansouri, Zhenyu Zhu

safety

arXiv Machine Learning

Oct 15

Structure-aware Bayesian full waveform inversion via diffusion-regularized Stein variational gradient descent

arXiv:2610.19187v1 Announce Type: cross Abstract: Full waveform inversion (FWI) is a powerful tool for high-resolution subsurface model reconstruction, but it remains highly nonlinear and ill-posed,...

By Chao Li, Mrinal K. Sen, Sergey Fomel, Yangkang Chen

diffusion safety

arXiv Machine Learning

Oct 15

FUSE: Fairness-aware Update Screening against Byzantine Exploitation in $q$-Fair Federated Learning

arXiv:2610.20264v1 Announce Type: new Abstract: Client-level fair federated learning, in the form of $q$-fair federated learning ($q$-FFL), agnostic federated learning, or tilted empirical risk minim...

By Tiantong Sun, Jinlin Liu, Xingru Wei, Zequan Fang, Wei Chen, Qiang Yang, Xiaojin Zhang

benchmarks safety

arXiv Machine Learning

Oct 15

SpectrumQuant: Spectrum Alignment for LLM Quantization

arXiv:2610.20303v1 Announce Type: new Abstract: Orthogonal transforms have emerged as an effective tool for low-bit quantization of large language models (LLMs), as they reshape activation distributi...

By Dwaipayan Haldar, Indranil Patra, Azhar Yousuf, Chandra Sekhar Seelamantula

llms efficiency benchmarks safety

arXiv Machine Learning

Oct 15

Predicting Unseen Scientific Interactions: Conditional Flow Models Hallucinate Non-Additive Effects

arXiv:2610.20288v1 Announce Type: new Abstract: Conditional generative models are increasingly used for scientific prediction beyond the direct training support. Yet scientific questions often concer...

By Sam Gijsen, Moritz Seiler, Kerstin Ritter

safety

Get the daily briefing