The Token Games (TTG) is an evaluation framework that pits language models against each other by having them create and solve programming puzzles. Using pairwise duels and Elo ratings, the authors benchmarked ten frontier models, achieving rankings comparable to established tests while spending under $200 and requiring no human puzzle creation. The study highlights that generating high-quality puzzles remains difficult for current models and proposes a new paradigm for reasoning evaluation that also tests creativity and task creation.
By Simon Henniger, Gabriel Poesia
arXiv:2508. 09883v2 Announce Type: replace-cross Abstract: Large language models (LLMs) demonstrate remarkable reasoning capabilities in tasks such as algorithmic coding and mathematical problem-solving.
By Xiaojun Wu, Xiaoguang Jiang, Huiyang Li, Jucai Zhai, Dengfeng Liu, Qiaobo Hao, Huang Liu, Zhiguo Yang, Ji Xie, Ninglun Gu, Jin Yang, Kailai Zhang, Yelun Bao, Jun Wang
arXiv:2609.14473v1 Announce Type: new
Abstract: Personal AI assistants hold the potential to evolve from digital interfaces into embodied companions capable of guiding users through complex physical...
By Avijit Dasgupta, Shayon Dasgupta, Zakaria Laskar, C. V. Jawahar, Karteek Alahari
arXiv:2604. 03374v2 Announce Type: replace-cross Abstract: Creative problem-solving requires combining multiple cognitive abilities, including logical reasoning, lateral thinking, analogy-making, and commonsense knowledge, to discover insights that connect seemingly unrelated pieces of information.
By Mete Ismayilzada, Renqing Cuomao, Daniil Yurshevich, Anna Sotnikova, Lonneke van der Plas, Antoine Bosselut
WordPolo is a word‑finding task that evaluates language models by having them guess an unknown target word and receive semantic similarity feedback. Participants start with no knowledge, make iterative guesses, and receive distance scores that guide them through semantic space. The study tests recent LLMs, LRMs, humans, and a heuristic on 1,500 puzzles, revealing that while solve rates vary widely, many models make meaningful progress and exhibit human‑like strategies, highlighting the importance of assessing reasoning processes, not just final accuracy.
By Tyler McDonald, Ali Emami
arXiv:2606. 07108v1 Announce Type: new Abstract: Recent advances in Large Reasoning Models (LRMs) demonstrate remarkable performance improvements by iteratively reflecting, exploring, and executing complex tasks, yet suffer from inefficiencies due to redundant reasoning, known as "overthinking".
By Tengyao Tu, Yulin Li, Hui-Ling Zhen, Libo Qin, Zhoujun Wei, Jinghua Piao, Zhuotao Tian, Yong Li, Min Zhang
arXiv:2606. 03883v1 Announce Type: new Abstract: Large reasoning models (LRMs) are often evaluated using metrics such as final-answer accuracy or token count.
By Fr\'ed\'eric Berdoz, Luca A. Lanzend\"orfer, Fabian Farestam, Roger Wattenhofer
arXiv:2512. 11995v2 Announce Type: replace-cross Abstract: While many vision-language models (VLMs) are developed to answer well-defined, straightforward questions with highly specified targets, as in most benchmarks, they often struggle in practice with complex open-ended tasks, which usually require multiple rounds of exploration and reasoning in the visual space.
By Chenrui Fan, Yijun Liang, Shweta Bhardwaj, Kwesi Cobbina, Ming Li, Tianyi Zhou
Selection-Based Structured Reasoning (SSR) is a framework that replaces free-form reasoning with the selection of pre-specified natural‑language reasoning candidates, allowing multimodal agents to choose from a set of reusable high‑level reasoning traces. By scoring these candidates in parallel using a shared context KV cache, SSR eliminates the need for an auxiliary task head and reduces inference cost. Experiments on seven multimodal search benchmarks with 2B and 4B models show that SSR maintains competitive success rates while cutting per‑turn reasoning latency by over 90% and overall inference latency by 28–54%.
By Feiyu Gavin Zhu, Xiaoyu Zhu, Jiqi Yang, Rui Yang, Arnab Kumar Mondal, Yancheng Wang, Xinke Deng, Jean Oh, Reid Simmons, Joerg Liebelt, Xiang Kong, Zhongyu Jiang
arXiv:2609.10445v1 Announce Type: new
Abstract: Reasoning language models have made substantial advances on a variety of complex tasks, yet their capabilities remain overwhelmingly English-centric: m...
By Mehrnaz Mofakhami, Ananya Sahu, Alejandro R. Salamanca, Daniel D'souza, Alexandre Berard, Thomas Euyang, Marzieh Fadaee, Julia Kreutzer
arXiv:2608. 10444v1 Announce Type: cross Abstract: Large language models (LLMs) have made substantial progress on reasoning tasks that require increasingly long and complex inferential chains.
By Si'an Xie (Beijing University of Posts and Telecommunications), Jiaxun Liu (Peking University), Biao Yang (Kuaishou Technology), Wei Yuan (Kuaishou Technology), Fan Yang (Kuaishou Technology), Tingting Gao (Kuaishou Technology), Ming Wu (Beijing University of Posts and Telecommunications)
InternBootcamp is an open‑source framework that offers over 1,000 domain‑diverse task environments for large language model (LLM) reasoning research. It introduces Bootcamp‑Eval, an automatically generated benchmark for comprehensive performance assessment. Experiments show that training on InternBootcamp significantly improves reasoning performance, with a 32B model achieving state‑of‑the‑art results on Bootcamp‑Eval and other established benchmarks, demonstrating that scaling the number of training tasks yields consistent gains.
By Peiji Li, Jiasheng Ye, Yongkang Chen, Linyang Li, Yichuan Ma, Zijie Yu, Ganqu Cui, Haozhan Li, Jiacheng Chen, Chengqi Lyu, Wenwei Zhang, Qipeng Guo, Dahua Lin, Bowen Zhou, Kai Chen