← Back to all news

AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,798 stories · RSS feed

arXiv Machine Learning
Oct 8

Restricting the Model, Missing the System: Measurement and Accountability in Offensive AI Governance

arXiv:2605.09504v3 Announce Type: replace-cross Abstract: We show that the instruments used to measure AI offensive capability fail in two ways: (i) they overstate harm, and (ii) they credit the mode...

By Michael Alexander Riegler, Finn Schwall, Annika Willoch Olstad, Birk Sebastian Frostelid Torpmann-Hagen, Sushant Gautam, Klas H. Pettersen, Inga Str\"umke
llmsagentssafety
More like this →
arXiv Machine Learning
Oct 8

Latent Performance Profiling of Large Language Models

arXiv:2605.30018v3 Announce Type: replace-cross Abstract: Large language models (LLMs) frequently achieve impressive scores on standardized benchmarks, yet accuracy alone offers a limited view of the...

By Tanmoy Chakraborty, Ayan Sengupta, Suparna Bhattacharya, Partha Pratim Chakrabarti, Amlan Chakrabarti, Supratik Chakraborty, Partha Pratim Das, Lipika Dey, Richa Singh, Mayank Vatsa
llmsbenchmarkssafety
More like this →
arXiv Machine Learning
Oct 8

From Prompts to Trees: Effective LLM-Guided Tree Generation for Few-Shot Tabular Classification

The paper introduces a three‑stage framework that uses Large Language Models (LLMs) to generate rules and assemble them into decision trees for few‑shot tabular classification. By distilling LLM knowledge into interpretable trees, the method avoids the high inference costs and limited interpretability of direct LLM application while outperforming traditional decision trees in low‑data settings. Experiments on real‑world datasets show superior accuracy and interpretability with lower prompting overhead compared to existing baselines.

By Yue Qiu, Zekang Du, Yiqun Diao, Bingsheng He, Qinbin Li
llmssafety
More like this →
arXiv Machine Learning
Oct 8

PatchBench: Measuring Collateral Damage in Activation Patching

PatchBench is a new benchmark designed to evaluate the collateral damage of activation patching in large language models, particularly for jailbreak repairs. It curates 400 high‑confidence jailbreak failures from 27,870 prompts across 37 datasets and introduces PatchBench‑Local, an evaluation protocol that tests whether a patch corrects harmful prompts while preserving benign ones. The study shows that aggregate metrics can miss severe local regressions, highlighting the need for more precise evaluation methods.

By Alexi Canesse, Mathis Le Bail, Ma\"el Jenny, Cl\'ement Elliker, Mahammed El Sharkawy, Sonia Vanier
llmsbenchmarkssafety
More like this →
arXiv Machine Learning
Oct 8

Shared Gaussianization: What Gaussian Regularizers Certify About Contrastive Learning, and What They Miss

The paper introduces Shared Gaussianization (SG), a characteristic‑function test that evaluates Gaussianity of the average of two normalized views scaled by an independent χ_d radius. SG detects both misalignment and non‑uniformity in contrastive learning, vanishing exactly at aligned, uniform minimizers of population InfoNCE, and provides a sharp, dimension‑free bound on InfoNCE excess. The study also explores how SG behaves under different kernels, alignment weights, batch sizes, and latent‑variable models, revealing optimization gaps and the impact of nuisance channels.

By Ruoyu Zhao, Yuting Chen, Jinheng Zhang, Zhehao Zou, Tong Che
ragsafety
More like this →
arXiv Computer Vision
Oct 8

From Global Alignment to Local Grounding: Zero-Shot Chinese Character Recognition with Radical Verification

arXiv:2610.09449v1 Announce Type: new Abstract: Zero-shot Chinese character recognition (ZS-CCR) aims to recognize characters whose categories are never observed during training, and typically relies...

By Yu-Heng Shih, Bing-Chen Wu, Tsz-To Wong, Ting-En Yen, Hong-Han Shuai, Bin-Hua Hsieh, Chien-An Chen, Yi-Ren Yeh, Ching-Chun Huang
ragbenchmarkssafety
More like this →