arXiv Machine Learning

Mitigating Factual Hallucination in Large Reasoning Models via Mixed-Mode Advantage Regularization

arXiv:2607. 05861v1 Announce Type: cross Abstract: Large reasoning models (LRMs) improve language model capabilities by generating explicit thinking traces before final answers.

arXiv AI

Aug 11

REIN: Bridging the Gap between Reasoning and Reliability via Reflection and Abstention Alignment

arXiv:2608. 07931v1 Announce Type: new Abstract: Large reasoning models (LRMs) are prone to hallucination, which undermines their reliability and poses challenges for safe deployment.

By Zhengze Huang, Luyang Yu, Di Hong, Xinzhe Huang, Wanyu Lin, Zhixuan Chu, Zhan Qin, Tianhang Zheng

benchmarks safety

arXiv AI

Jun 10

TruthRL: Incentivizing Truthful LLMs via Reinforcement Learning

arXiv:2509. 25760v2 Announce Type: replace-cross Abstract: While large language models (LLMs) have demonstrated strong performance on factoid question answering, they are still prone to hallucination and untruthful responses, particularly when tasks demand information outside their parametric knowledge.

By Zhepei Wei, Xiao Yang, Kai Sun, Jiaqi Wang, Rulin Shao, Jingxiang Chen, Mohammad Kachuee, Teja Gollapudi, Yiwei Liao, Nicolas Scheffer, Rakesh Wanga, Anuj Kumar, Yu Meng, Wen-tau Yih, Xin Luna Dong

llms nlp reinforcement-learning benchmarks safety

arXiv AI

2d ago

The Metacognitive Bottleneck: Japanese Riddles Reveal Fundamental Limits of Machine Insight and Self-Evaluation in Reasoning AI

arXiv:2509. 14704v3 Announce Type: replace Abstract: Benchmark saturation and training-data contamination increasingly obscure whether reported gains in large language models (LLMs) reflect genuine advances in reasoning or familiarity with recurring patterns in benchmark problems.

By Masaharu Mizumoto, Dat Nguyen, Zhiheng Han, Xingfu Li, Yo Nakawake, Le Minh Nguyen

llms benchmarks

arXiv AI

Aug 3

On the Generalization of Steering Vectors for Chain-of-Thought Faithfulness

arXiv:2607. 29062v1 Announce Type: new Abstract: Model capabilities have improved in large part due to scaling chain of thought.

By Matthew Nguyen, Kyle Cox, Austin Meek, Iv\'an Arcuschin

safety

arXiv AI

Jun 2

An Enigma of Artificial Reason: Investigating the Production-Evaluation Gap in Large Reasoning Models

arXiv:2606. 01462v1 Announce Type: new Abstract: Studies of human reasoning have shown that people are typically stronger at evaluating reasoning than producing it from scratch.

By Mingzhong Sun, Teresa Yeo, Armando Solar-Lezama, Tan Zhi-Xuan

safety

arXiv AI

Jul 24

Post-Hoc Reasoning in Chain of Thought: Decoding and Steering Pre-Committed Answers

arXiv:2603. 01437v2 Announce Type: replace Abstract: As chain of thought (CoT) has become central to scaling reasoning capabilities in large language models (LLMs), it has also emerged as a promising tool for interpretability, suggesting the opportunity to understand model decisions through verbalized reasoning.

By Kyle Cox, Darius Kianersi, Adri\`a Garriga-Alonso

llms safety

arXiv Machine Learning

Aug 11

Rethinking Reasoning with MDLMs: Early Exits, Post-hoc Reasoning, and Beyond

arXiv:2510. 19990v2 Announce Type: replace Abstract: The reasoning paradigm, where language models reason before answering, has enabled breakthroughs on tasks such as mathematical problem-solving.

By Zachary Horvitz, Raghav Singhal, Hao Zou, Carles Domingo-Enrich, Zhou Yu, Rajesh Ranganath, Kathleen McKeown

llms diffusion reinforcement-learning fine-tuning

arXiv AI

Aug 11

Understanding and Mitigating Premature Confidence for Better LLM Reasoning

arXiv:2605. 24396v2 Announce Type: replace Abstract: Long chains of thought (CoT) from current language models frequently contain logical gaps and unjustified leaps, limiting the gains from additional test-time compute.

By Jingchu Gai, Guanning Zeng, Christina Baek, Chen Wu, J. Zico Kolter, Andrej Risteski, Aditi Raghunathan

llms reinforcement-learning benchmarks safety

arXiv AI

Jun 9

Thinking-Based Non-Thinking: Solving the Reward Hacking Problem in Training Hybrid Reasoning Models via Reinforcement Learning

arXiv:2601. 04805v2 Announce Type: replace Abstract: Large reasoning models (LRMs) have attracted much attention due to their exceptional performance.

By Siyuan Gan, Jiaheng Liu, Boyan Wang, Tianpei Yang, Runqing Miao, Yuyao Zhang, Fanyu Meng, Junlan Feng, Linjian Meng, Jing Huo, Yang Gao

reinforcement-learning fine-tuning benchmarks

arXiv Machine Learning

Jun 2

Towards Lightweight Reliability: Using Soft Prompts for Hallucination Mitigation in Large Language Models

arXiv:2606. 00919v1 Announce Type: cross Abstract: Large language models (LLMs) have seen widespread adoption across various domains, yet their reliability is frequently undermined by hallucinations - responses that are plausible-sounding but factually incorrect.

By S M Tahmid Siddiqui, Akib Jawad Ononto, Anoop Singhal, Latifur Khan

llms safety

arXiv AI

Jun 11

From Architecture to Output: Structural Origins of Hallucination in Large Language Models and the Amplifying Role of Data

arXiv:2606. 07537v1 Announce Type: cross Abstract: Large language models hallucinate--producing fluent, confident, factually wrong outputs--with a consistency that persists across generations and scales.

By Md. Rejaul Korim Sadi, Toufiqur Rahman Tasin, Golam Mostofa Naeem

llms safety

arXiv AI

Jun 30

Reconsidering Overthinking: Penalizing Internal and External Redundancy in CoT Reasoning

arXiv:2508. 02178v3 Announce Type: replace Abstract: Large reasoning models (LRMs) often exhibit overthinking, producing verbose Chain-of-Thought (CoT) traces that increase inference cost and obscure the underlying reasoning process.

By Taihang Zhen, Jialiang Hong, Kai Chen, Guang Yang, Junlan Feng, Wenpeng Zhu, Jing Huo, Yang Gao, Depeng Wang, Haitao Wan, Xi Yang, Fanyu Meng, Yuyao Zhang, Ji Qi, Xiangyu Zhou

reinforcement-learning

Related stories