arXiv AI

Geometry of Divergence: Tracking Hidden-State Trajectories for Adaptive Multi-Turn Reasoning

arXiv Machine Learning
Jun 9

Attention Illuminates LLM Reasoning: The Preplan-and-Anchor Rhythm Enables Fine-Grained Policy Optimization

arXiv:2510. 13554v2 Announce Type: replace-cross Abstract: The reasoning pattern of Large language models (LLMs) remains opaque, and reinforcement learning (RL) typically applies uniform credit across an entire generation, blurring the distinction between pivotal and routine steps.

By Yang Li, Zhichen Dong, Yuhan Sun, Weixun Wang, Shaopan Xiong, Yijia Luo, Jiashun Liu, Han Lu, Jiamang Wang, Wenbo Su, Bo Zheng, Junchi Yan
arXiv Machine Learning
Aug 7

STATe-of-Thoughts: Structured Action Templates for Tree-of-Thoughts

arXiv:2602. 14265v3 Announce Type: replace-cross Abstract: Inference-Time-Compute (ITC) methods like Best-of-$n$ and Tree-of-Thoughts are meant to produce output candidates that are both high-quality and diverse, but their use of high-temperature sampling often fails to achieve meaningful output diversity.

By Zachary Bamberger, Till R. Saenger, Gilad Morad, Ofra Amir, Brandon M. Stewart, Amir Feder
arXiv AI
Jun 24

Beyond Trajectory Imitation: Strategy-Guided Policy Optimization for LLM Reasoning

arXiv:2606. 24064v1 Announce Type: new Abstract: Distilling reasoning capabilities from strong to weak language models typically involves imitating specific solution trajectories, effectively transferring what to answer rather than how to reason.

By Tianyuan Shi, Canbin Huang, Bei Li, Xin Chen, Xiaojun Quan, Jingang Wang, Qifan Wang
arXiv AI
Sep 10

Reasoning Depth and Environment Complexity: A Controlled Study of RLVR Data Allocation across Logical Reasoning Tasks

The paper investigates reinforcement learning with verifiable rewards (RLVR) by expanding the reasoning space beyond depth to include environment complexity and diverse reasoning forms. It introduces a synthetic knowledge‑graph environment that varies depth, complexity, and task family, revealing that joint depth‑complexity coverage outperforms single‑axis approaches, that different reasoning families behave non‑uniformly, and that uniform mixing beats staged curricula under a fixed budget. The study also shows that current off‑the‑shelf models share a deductive‑over‑abductive bias, indicating a broader gap in reasoning capabilities.

By Yihua Zhu, Qianying Liu, Fei Cheng, Jiaxin Wang, Akiko Aizawa, Sadao Kurohashi, Hidetoshi Shimodaira