ChaosBench-Logic is a benchmark that tests large language models on logical and symbolic reasoning within chaotic dynamical systems. It covers 30 diverse systems, each annotated with 11 semantic predicates, and generates 621 questions across seven reasoning categories such as multi-hop implications, cross-system analogies, and counterfactual reasoning. The benchmark defines metrics for logical accuracy, implication consistency, dialogue coherence, and contradiction, and provides an open-source evaluation pipeline.
By Noel Thomas
arXiv:2607. 18100v1 Announce Type: new Abstract: Extended reasoning has become standard for frontier Large Language Models (LLMs), yet the trajectories these models produce remain largely uncontrollable.
By Sheldon Yu, Tong Yu, Xunyi Jiang, Rohan Surana, Gagan Mundada, Sungchul Kim, Lina Yao, Julian McAuley, Junda Wu
arXiv:2605. 22873v2 Announce Type: replace-cross Abstract: Chain-of-thought (CoT) reasoning has become the default strategy for enhancing LLM capabilities, yet its application raises a fundamental question: when is explicit reasoning actually beneficial?
By Wei Xia, Haoqing Wang, Zhi-Hong Deng, Yehui Tang
arXiv:2510. 27544v3 Announce Type: replace Abstract: Current training paradigms, optimized for long-horizon reasoning trace execution, have made Large Language Models (LLMs) excel at pattern matching and forward simulation of reasoning, but underperform at counterfactual causal understanding and reasoning.
By Nikolaus Holzer, William Fishell, Baishakhi Ray, Mark Santolucito
The paper investigates how monitoring the internal representations of Large Reasoning Models (LRMs) during their Chain-of-Thought (CoT) reasoning can predict future behavior. By evaluating a probe at each generated token, the authors construct probe trajectories that capture the continuous evolution of a concept’s probability, and extract signal‑processing features to characterize volatility, trend, and steady‑state behavior. These trajectory features improve predictive performance over static probes, adding an average of 21.7 AUROC points, and the study demonstrates that template‑based training data and pooling operations are essential for effective monitoring across multiple reasoning models and datasets.
By Maciej Chrabaszcz, Aleksander Szymczyk, Marcin Sendera, Tomasz Trzcinski, Sebastian Cygert
arXiv:2607. 08017v1 Announce Type: cross Abstract: Large-Language Models (LLMs) can be prone to flawed and unfaithful reasoning that decoding strategies like Self-Consistency (SC) fail to detect as they evaluate only final-answer agreement while ignoring the logical validity of intermediate steps.
By Riccardo Revalor, Jalees Rehman, Debjit Pal