arXiv Machine Learning August 31, 2026 By ShengYun Peng, Pin-Yu Chen, Eric Smith, Song Jiang, Hongyuan Zhan, Haozhu Wang, Mahesh Pasupuleti, Duen Horng Chau, Jianfeng Chi

Large Reasoning Models Learn Better Alignment from Flawed Thinking

Read the original on arXiv Machine Learning →

Large reasoning models generate chain-of-thought (CoT) before answering but struggle with safety alignment and can be misled by flawed premises. The paper introduces RECAP, a reinforcement learning approach that trains models to override flawed reasoning paths and produce safe, helpful responses without extra training cost. RECAP improves safety, jailbreak robustness, and reduces overrefusal while preserving core reasoning abilities and inference token budget.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Jul 7

Oyster-II: Reinforcement Learning for Constructive Safety Alignment in Large Language Models

arXiv:2607. 02914v1 Announce Type: new Abstract: Large language models (LLMs) have demonstrated remarkable capabilities across diverse applications, yet ensuring their simultaneous safety, helpfulness, and trustworthiness remains a persistent challenge.

By Jiyang Guan, Yong Xie, Jun Chen, Jiexi Liu, Zipeng Ye, Defeng Li, Jiayu Shen, Jialing Tao, Hui Xue

llms reinforcement-learning fine-tuning benchmarks safety

arXiv Machine Learning

Jun 2

Enhancing LLM Metacognition via Cognitive Pairwise Training

arXiv:2606. 00869v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) has become central to LLM reasoning, but its outcome-level rewards can make models more willing to give confident answers when evidence or reasoning is unreliable.

By Weitao Li, Hao Zhou, Xuanyu Lei, Fandong Meng, Yuanhang Liu, Jingyi Ren, Ante Wang, Xiaolong Wang, Yuanchi Zhang, Fuwen Luo, Guangwen Yang, Lin Gan, Weizhi Ma, Yang Liu

llms reinforcement-learning fine-tuning safety

arXiv AI

Jun 30

Mitigating the Safety-utility Trade-off in LLM Alignment via Adaptive Safe Context Learning

arXiv:2602. 13562v2 Announce Type: replace-cross Abstract: While reasoning models have achieved remarkable success in complex reasoning tasks, their increasing power necessitates stringent safety measures.

By Yanbo Wang, Minzheng Wang, Jian Liang, Lu Wang, Yongcan Yu, Ran He

llms reinforcement-learning efficiency safety

arXiv AI

Aug 5

Self-Guided Adaptive Safety Alignment: Synthesizing and Internalizing Guidelines in Reasoning Models

arXiv:2511. 21214v4 Announce Type: replace-cross Abstract: Explicit safety policies can improve reasoning-model safety, but their effective coverage may lag behind evolving jailbreak strategies.

By Yuhang Wang, Yanxu Zhu, Jiaming Zhang, Dongyuan Lu, Jitao Sang

safety

arXiv AI

Jul 7

CAP-CoT: Cycle Adversarial Prompt for Improving Chain of Thoughts in LLM Reasoning

arXiv:2604. 23270v2 Announce Type: replace Abstract: Chain-of-Thought (CoT) prompting has emerged as a simple and effective way to elicit step-by-step solutions from large language models (LLMs).

By Shuxu Chen, Yitian Zhou, Jiaquan Zhang, Haoyu Bian, Wenrui Hu, Aming Wu, Sungyoung Lee, Chaoning Zhang, Hyundong Shin

llms agents benchmarks safety

arXiv Machine Learning

Jul 1

Addressing Over-Refusal in LLMs with Competing Rewards

arXiv:2606. 31748v1 Announce Type: new Abstract: Safety training on language models often induces over-refusal: improved safety on harmful prompts at the cost of increased refusal on harmless ones.

By Taeyoun Kim, Aviral Kumar

llms reinforcement-learning safety

One email a morning, machine-written

Related stories

Oyster-II: Reinforcement Learning for Constructive Safety Alignment in Large Language Models

Enhancing LLM Metacognition via Cognitive Pairwise Training

Mitigating the Safety-utility Trade-off in LLM Alignment via Adaptive Safe Context Learning

Self-Guided Adaptive Safety Alignment: Synthesizing and Internalizing Guidelines in Reasoning Models

CAP-CoT: Cycle Adversarial Prompt for Improving Chain of Thoughts in LLM Reasoning

Addressing Over-Refusal in LLMs with Competing Rewards