The paper surveys efficient reasoning in large language models, contrasting fast intuitive (System 1) and slow deep (System 2) reasoning. It analyzes why System 2 is computationally costly yet more accurate, and why System 1 is efficient but less effective. The survey covers causes of inefficiency, patterns of reasoning behavior, and potential solutions to balance performance and computational budgets, offering actionable insights and an open‑source repository for ongoing research.
By Rui Wang, Hongru Wang, Boyang Xue, Jianhui Pang, Shudong Liu, Yi Chen, Jiahao Qiu, Derek Fai Wong, Heng Ji, Kam-Fai Wong
arXiv:2505. 12992v4 Announce Type: replace-cross Abstract: Inference-time scaling techniques have significantly bolstered the reasoning capabilities of large language models (LLMs) by harnessing additional computational effort at inference without retraining.
By Baohao Liao, Hanze Dong, Yuhui Xu, Doyen Sahoo, Christof Monz, Junnan Li, Caiming Xiong
LongZero is a reinforcement learning method that enables large language models to improve long‑context reasoning without external data. It works by having the model first generate a compact, structured question‑answer pair with supporting evidence, then expanding that evidence into a long, multi‑document context. The approach combines self‑written multi‑hop reasoning and multi‑round retrieval tasks, trained with GRPO, and outperforms baseline RL methods on five long‑context reasoning benchmarks.
By Zhennan Wan, Jianfei Chen
arXiv:2505. 17315v2 Announce Type: replace Abstract: Recent language models exhibit strong reasoning capabilities, yet the influence of long-context capacity on reasoning remains underexplored.
By Wang Yang, Zirui Liu, Hongye Jin, Qingyu Yin, Vipin Chaudhary, Xiaotian Han
The paper introduces ProloNg, a synthetic benchmark designed to test long-horizon deductive reasoning in language models using Prolog. It evaluates eight reasoning models from five leading LLM families on problems with reasoning depths up to 22 and context lengths of 62,000 tokens. Results show that model performance drops sharply as reasoning depth increases, with most models performing near chance beyond a depth of 10.
By Hadeel Al-Negheimish, Jasna Ilieva, Yoon Kim
arXiv:2603. 08000v2 Announce Type: replace-cross Abstract: Large reasoning models (LRMs) like OpenAI o1 and DeepSeek-R1 achieve high accuracy on complex tasks by adopting long chain-of-thought (CoT) reasoning paths.
By Chenzhi Hu, Qinzhe Hu, Yuhang Xu, Junyi Chen, Ruijie Wang, Shengzhong Liu, Jianxin Li, Fan Wu, Guihai Chen