The paper introduces a task‑centric framework that analyzes the internal processing of Large Language Models (LLMs) by combining attention, activation, and representation signals. It evaluates 16 LLMs on 24 benchmarks, revealing systematic differences between knowledge‑centric and reasoning‑centric tasks and exposing failure modes hidden by overall scores. The authors provide open‑source code to support more targeted benchmark selection and informed model choice.
By Yang Wang, Xiao Wang, Yiqi Liu, Chenghao Xiao, Eleni Vasilaki, Goran Nenadic, Chenghua Lin
arXiv:2608.30426v1 Announce Type: new
Abstract: Current dialogue systems struggle with dynamic information retrieval, often leading to hallucinations and lower response accuracy. We address this by a...
By Markel Ferro, Oier Lopez de Lacalle
The paper investigates how transformers acquire deep semantic dependencies, proposing a mechanistic framework that frames learning as a competition between surface statistics and deep semantics. It identifies a "Gradient Starvation" effect that suppresses error signals for sparse semantic dependencies early in training, delaying structural reasoning until a sudden phase transition. The study also explains the success of Chain-of-Thought strategies and introduces a topology‑aligned contrastive objective that improves variable binding performance by more than twice the gain of standard fine‑tuning.
By Jiangrui Zhao, Xiaoting Du
arXiv:2607. 06160v1 Announce Type: cross Abstract: Synthesizing long-context supervised fine-tuning (SFT) data is a scalable way to enhance the long-context understanding of large language models (LLMs), yet existing approaches share three limitations: narrow task coverage, insufficient instruction difficulty, and a lack of faithfulness supervision.
By Chenhao Yuan, Yinhao Xu, Shuwen Xu, Xizhi Yang, Jiaxiang Liu, Chenxi Zhou, Shaoping Huang, Haolin Ren, Pengfei Cao, Jun Zhao, Kang Liu
arXiv:2510.03223v3 Announce Type: replace-cross
Abstract: Large Language Models (LLMs) tend to generate a long reasoning chain when solving complex tasks. However, as the reasoning chain extends, cri...
By Hongxiang Zhang, Yuan Tian, Tianyi Zhang
The paper surveys efficient reasoning in large language models, contrasting fast intuitive (System 1) and slow deep (System 2) reasoning. It analyzes why System 2 is computationally costly yet more accurate, and why System 1 is efficient but less effective. The survey covers causes of inefficiency, patterns of reasoning behavior, and potential solutions to balance performance and computational budgets, offering actionable insights and an open‑source repository for ongoing research.
By Rui Wang, Hongru Wang, Boyang Xue, Jianhui Pang, Shudong Liu, Yi Chen, Jiahao Qiu, Derek Fai Wong, Heng Ji, Kam-Fai Wong
The paper introduces Difficulty-Aware Chain-of-Thought (DA‑CoT), a training‑free framework that selectively applies chain‑of‑thought prompting in reasoning segmentation tasks based on the entropy of attention maps. Low entropy signals confident, localized grounding, while high entropy indicates uncertainty and the potential benefit of additional reasoning. DA‑CoT also incorporates a Bidirectional Rethinking mechanism to automatically reverse reasoning decisions when initial outputs fail, thereby improving robustness and inference efficiency while achieving state‑of‑the‑art results on Reasoning Segmentation benchmarks.
By Chanseul Cho, Jaesung Jun, Dongheon Lee, Youngmin Ro
The paper surveys recent progress in supervised fine‑tuning (SFT) for large language models that excel at reasoning tasks. It reviews methodological advances, grouping them into data‑centric and algorithm‑centric approaches, and reorganizes analytical studies by data characteristics, optimization dynamics, and mechanistic insights. The survey also identifies current research focuses, remaining bottlenecks, and suggests future directions for reasoning SFT.
By Yongcan Yu, Kuangpu Guo, Jian Liang, Lingxiao He, Meng Wang, Ran He
The paper investigates how different forms of compressed chain‑of‑thought (CoT) reasoning—Explicit, Composed, and Implicit—affect large language model (LLM) performance after supervised fine‑tuning (SFT). Using a synthetic compositional reasoning task, the authors show that coarser CoT requires more SFT data, that Composed and Implicit CoT benefit more from data scaling (with Composed also benefiting from repetition), and that reinforcement learning with verifiable rewards (RLVR) can decompose compressed steps learned during SFT. Additionally, unidirectional CoT ordering improves generalization on longer sequential tasks.
By Kohsei Matsutani, Gouki Minegishi, Takeshi Kojima, Yusuke Iwasawa, Yutaka Matsuo
arXiv:2604. 01161v2 Announce Type: replace Abstract: Large language models (LLMs) exhibiting test-time scaling behavior, such as extended reasoning traces and self-verification, have demonstrated remarkable performance on complex, long-term reasoning tasks.
By Gleb Rodionov, Roman Garipov, George Yakushev
LongReasonArena is a new benchmark that evaluates the long reasoning abilities of large language models, rather than just their comprehension of long inputs. It presents tasks that require multi-step algorithms involving retrieval and backtracking, with reasoning lengths that can be scaled up to one million tokens. Evaluation shows that even advanced models like GPT‑5 struggle, achieving only 27.8% accuracy on the hardest level, and accuracy decreases linearly with the logarithm of expected reasoning steps.
By Jiayu Ding, Shuming Ma, Lei Cui, Nanning Zheng, Furu Wei
arXiv:2510. 01427v3 Announce Type: replace Abstract: At the core of Deep Research is knowledge mining, the task of extracting structured information from massive unstructured text in response to user instructions.
By Sipeng Zhang, Shuhuai Lin, Xinpeng Wei, Yihang Chen, Pin Qian, Su Wang, Huan Xu