arXiv AI

When the Chain of Thought Knows Better: Failure Modes in Multi-Turn Reasoning Models

arXiv:2606. 10740v1 Announce Type: new Abstract: Failures in multi-turn reasoning models are largely invisible to terminal-score evaluation.

arXiv AI
Sep 17

First Token Matters: Understanding Safety Collapse in Large Reasoning Models

The paper investigates why large reasoning models (LRMs) lose safety alignment when faced with harmful queries. By analyzing token-level refusal dynamics, the authors identify a vulnerability called Onset Refusal Collapse (ORC), where the refusal signal drops sharply at the first generated token, leading to unsafe responses. They introduce SafeToken, a lightweight inference-time intervention that injects a learned safety anchor at reasoning onset, which mitigates ORC, improves safety on harmful-query benchmarks, and largely preserves reasoning utility.

By Yizheng Yang, Haining Yu, Yuechen Wang, Yikai Hou, Xing Fu, Jinbo Yang, Tianqing Zhu
arXiv AI
Aug 26

TRACE: An Evidence-Grounded Benchmark for Safety Evaluation of Large Reasoning Models

TRACE is a new benchmark that evaluates the safety of Large Reasoning Models (LRMs) across the entire inference pipeline, including prompts, reasoning traces, and final responses. It provides prompts in two languages covering nine risk categories and ten attack strategies, and for each prompt four LRMs generate traces and responses that are annotated for safety with supporting evidence extracted from the source text. Evaluation of 18 guardrail models on TRACE shows that detecting unsafe content in reasoning traces is much harder than in prompts or final responses, and that current models struggle to extract the necessary evidence.

By Zhenyu Wu, Siyuan Chen, Changchun Yang, Jiaqi Dong, Min Zhou, Ali Almadan, Talal Hammad, Faisal Wahbo, Aminullah Tora, Mona Alshahrani, Xin Gao