← Back to all news

AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,798 stories · RSS feed

arXiv Computer Vision
Oct 8

Efficient Dense Crowd Trajectory Prediction Via Dynamic Clustering

The paper introduces a cluster-based method for predicting dense crowd trajectories, grouping individuals with similar attributes over time to accelerate processing. By replacing individual pedestrian inputs with cluster centroids, the approach reduces computational cost and memory usage while preserving accuracy. Extensive evaluations on challenging dense crowd scenes confirm these benefits compared to state‑of‑the‑art methods.

By Antonius Bima Murti Wijaya, Paul Henderson, Marwa Mahmoud
benchmarkssafety
More like this →
arXiv Machine Learning
Oct 8

Open-Weight LLM Fine-Tuning Defenses are Susceptible to Simple Attacks

The paper examines the vulnerability of open‑weight large language model safeguards that aim to block harmful usage by assuming new harmful behavior arises only through fine‑tuning. It demonstrates that simple, low‑cost attacks—abliteration and prefilling—can significantly increase attack success rates across multiple benchmarks, and proposes an abliteration‑resistant tuning (ART) method that mitigates these attacks. The study shows that safeguarding defenses must be evaluated against a broader range of attack strategies beyond fine‑tuning.

By Kevin Kuo, Virginia Smith, Chhavi Yadav
llmsfine-tuningbenchmarkssafety
More like this →
arXiv Machine Learning
Oct 8

Conditional Accuracy Profiles: Diagnosing LLM Judges across Deployment Conditions

The paper introduces Conditional Accuracy Profiling (CAP), a diagnostic framework that breaks down LLM‑judge accuracy into eight deployment conditions covering content sensitivity, robustness, and rationale quality. CAP is benchmark‑agnostic and can be applied directly when annotations are available, via task‑subset proxies, or through controlled augmentation. Applied to seven judges across six benchmarks, CAP reveals that aggregate accuracy masks important differences—for example, judges highly sensitive to omitted qualifications rank poorly on a Hard‑Constructed subset, and Position Robustness is generally stable but can degrade sharply under adversarial stress.

By Wenqi Li, Bin Liu, Mindi Ruan, Chuanbo Hu, Minglei Yin, Xin Li
llmsbenchmarkssafety
More like this →
arXiv Machine Learning
Oct 8

Efficient Best-of-N policy evaluation for inference-time alignment

The paper introduces a sample‑only framework for evaluating and selecting Best‑of‑N (BoN) inference‑time alignment policies without needing response likelihoods. By exploiting the order‑statistic structure of BoN, the authors express required density ratios through score‑rank probabilities that can be estimated from samples alone, and develop a doubly robust estimator (BoN‑DR) that reuses a shared auxiliary sample pool across candidate budgets. The method provides valid asymptotic inference even with reward estimator misspecification, proves efficiency, and offers two selection rules—maximizing estimated policy value or a lower confidence bound—to avoid over‑optimization and guarantee no harm, as demonstrated on synthetic data and GSM8K.

By Jonas Schweisthal, Yuxin Wang, Athiya Deviyani, Stefan Feuerriegel, Dennis Frauen
safety
More like this →
arXiv Machine Learning
Oct 8

In-Context Residual Calibration for Uncertainty Quantification of Energy Time Series over Graphs

The paper introduces STOIC, a post‑hoc residual calibration framework that enhances uncertainty quantification for energy time series over graphs. STOIC combines spatial‑temporal graph neural network forecasts with in‑context learning from tabular foundation models to estimate feature‑conditional residual quantiles, thereby capturing both sequential and relational dependencies. It is designed as a conformal‑inspired method that does not require task‑specific retraining of the calibration model.

By Keivan Faghih Niresi, Alice Cicirello, Olga Fink
llmssafety
More like this →
arXiv Computation and Language
Oct 8

PHRBench: A Behavioral Evaluation of Post-Hallucination Reasoning in LLMs

PHRBench is a new benchmark that evaluates how large language models handle hallucinated information during multi-stage reasoning. It tests 18 models across four domains using 4,820 controlled instances, measuring Hallucination Compliance, Hallucination Avoidance, and Heuristic Correction to determine if a model can correct hallucinated premises and still reach the correct answer. The study finds that successful recovery is rare, often requires frequent belief updates, and that features of the hallucinated prompt can predict recovery with an AUROC of 0.847.

By Linghao Meng, Feng He, Xuan Yang, Junyuan Mao, Pinze Ren, Deqing Mu, Hesen Yang, Qiankun Li
llmsbenchmarkssafety
More like this →
arXiv Machine Learning
Oct 8

Robust Decentralized Fairness Auditing

arXiv:2610.10199v1 Announce Type: new Abstract: Emerging legislation requires large language models (LLMs) to be audited for compliance with regulatory standards, particularly fairness. Such black-bo...

By Sayan Biswas, Jade Garcia Bourr\'ee, Anne-Marie Kermarrec, Palak, Martijn de Vos
llmssafety
More like this →