Safety-Flag is a unified benchmark that consolidates seven popular safety datasets into a single balanced flag/do‑not‑flag protocol, providing item‑level decisions and confidence scores for multiple large language models and dedicated guards. The benchmark evaluates moderator reliability across three dimensions—error direction, probability calibration, and confidence‑based error ranking—revealing that aggregate accuracy masks significant differences, such as one model flagging 85% of benign content while another misses 54% of harmful content. The study shows that general‑purpose models are overconfident, but temperature tuning can substantially improve calibration, and confidence‑based abstention can reduce selective risk, though performance varies with how well confidence ranks errors.
By Yibo Hu
arXiv:2610.27266v1 Announce Type: cross
Abstract: Preference labels align model training, but the judgments behind them are difficult to inspect or verify. In this paper, we examine how preference co...
By Chenhan Lyu, Amir Rahmani
The paper introduces a framework for evaluating AI systems that not only checks final labels but also tracks the reasoning behind them through three core sources—grounds, norms, and authority—forming an eight-cell counterfactual judgment cube. It defines minimal source replacement sets, called judgment receipts, to explain changes in verdicts and provides certification cost bounds for black-box evaluators. The authors present ReasonBench, a benchmark with 19,520 cases, and demonstrate that while high standard accuracy can mask robustness issues, receipt accuracy reveals significant gaps in reasoning consistency across different models.
By Ye Chen, Weining Zhang
arXiv:2606. 03650v1 Announce Type: cross Abstract: Choosing or ranking language models for a specific application is hardest when no task-specific labeled data exists, and standard public benchmarks cannot be trusted, their items having likely leaked into pretraining, so scores reflect memorization rather than fitness.
By Alexander Apartsin, Yehudit Aperstein
StepSafe introduces a training framework called ASPO that focuses on step‑level timely intervention in agentic systems. It aligns decisions with their respective prefixes, rewards actions that fit the current stage, and uses adaptive feedback self‑distillation to credit reasoning. The resulting StepSafe guardrail achieves state‑of‑the‑art performance on five agent‑safety benchmarks, improving the safety‑utility Pareto curve with a Youden’s J of 53.0 for a 4B model.
By Siyi Gu, Kunhan Gao, Chao Peter Yang, Sukanta Ganguly, Rex Ying, Arman Cohan
arXiv:2610.16814v1 Announce Type: cross
Abstract: Large language models (LLMs) are increasingly used as judges to assign scalar scores that drive downstream decision-making, including tasks such as p...
By Anning Hu, Yangyu Li, Yuqi Huang, Shengchao Hu, Junchi Yan
arXiv:2605. 23055v2 Announce Type: replace-cross Abstract: Frontier language models sometimes recognize that they are being evaluated and adjust their behavior, undermining validity of benchmark results.
By Changling Li, Terry Jingchen Zhang, Jie Zhang, Zhijing Jin, Sahar Abdelnabi, Maksym Andriushchenko
arXiv:2608. 03838v1 Announce Type: new Abstract: Reasoning-based guard models improve LLM safeguards, but decoding explicit rationales for every interaction makes them costly to deploy.
By Zhinan Liu, Jie Li, Mingyu Kang, Jiayi Ji
The paper introduces Value Probes, a method that uses linear probing on a language model’s activations to estimate a value function for future rewards in automated red‑team auditing. By treating the audit as a Markov decision process and applying beam search guided by these value estimates, the authors achieve a roughly 50% increase in audit success across 18 target behaviors and three model scales. The approach also demonstrates that Value Probes can be trained on off‑policy data from smaller models, offering a more efficient alternative to costly language‑model follow‑ups.
By Jonathan F. Jao, Andrew Kyle Lampinen
The paper introduces DiagSafe, a diagnostic evaluation framework that replaces binary safe/unsafe labels with prompt-level rubrics grounded in AI safety policies. DiagSafe uses a bilingual benchmark with an 84‑label safety taxonomy, comprising DiagBase and DiagAttack subsets, to assess safety and usefulness independently. Experiments show DiagSafe achieves high agreement with human annotations (95.34%) and strong inter‑judge consistency (Cohen’s κ = 0.864), revealing significant differences in unsafe rates and vulnerabilities across leading LLMs, especially in Chinese and multi‑turn contexts.
By Hongyu Li, Xinlong Huang, Zhan Yu, Jing Zhao, Zhe Zhao, Juntao Zhong, Xiong Kuang, Rui Yuan, ZhenXiang Yan, Huilin Xu, Haitao Yu, Lilin Wang, Rui Chen, Blaze Chen, Fengzong Lian, Renfen Hu
arXiv:2606. 09866v1 Announce Type: cross Abstract: Fine-tuning safety aligned large language models (LLMs) on downstream data improves adaptation but may erode learned safety behavior.
By Xinrui Chen, Jianhao Zhang, Ou Wu, Di Gao
arXiv:2610.15365v1 Announce Type: new
Abstract: Large language models (LLMs) can produce harmful content while generating a response. A useful safeguard should not only detect this risk during genera...
By Ankur Garg, Xuemin Yu, Hassan Sajjad, Samira Ebrahimi Kahou