Large language models confabulate chemical objects (molecular formulas, space groups, formation energies) in fluent reasoning traces, concentrated on long-tail entities where confidence is least trustworthy. Deterministic, database-grounded verification can catch and repair such errors without the coverage cost of blanket retrieval; the binding constraint, we find, is detection, not repair.
arXiv:2606. 03660v1 Announce Type: new Abstract: Large language models are increasingly used as chemistry assistants, yet most chemistry benchmarks still score only final answers.
By Hongyu Guo, Hao Li, He Cao, Gongbo Zhang, Li Yuan
arXiv:2607. 17047v1 Announce Type: cross Abstract: LLM constraint reasoners are often evaluated near the random-SAT phase transition, confounding density and solver hardness.
By Lucky Verma
arXiv:2606. 19808v1 Announce Type: new Abstract: Test-time reasoning is increasingly used as a serving-time control knob, but extra reasoning is not uniformly valuable: it can repair failed attempts, waste compute on already-correct answers, or introduce harmful answer changes.
By Sajib Acharjee Dip, Dawei Zhou, Liqing Zhang
The paper investigates whether large language models’ reasoning traces truly contain early, informative signals or merely reflect budget and difficulty confounds. Using a restart‑controlled truncation probe, the authors compare continuation success rates against from‑scratch restart curves across 178 problem‑model pairs, finding that only one case shows prefix‑limited success and that continuing a model’s own prefix generally outperforms restarting. A difficulty‑controlled test and two generation‑free analyses reveal that early internal signals do not carry outcome information beyond a problem‑difficulty baseline, underscoring the need for proper counterfactual controls.
By Yigit Utku Bulut
arXiv:2606. 27383v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as research assistants, yet it remains unclear whether they can calibrate research takeaways to the strength and scope of the supporting evidence.
By Yu Fu, Yongqi Kang, Yong Zhao
CORE is a search controller that uses a verifier to obtain a certified conflict core, backjumps to the latest decision in that core, and caches the conflict to prevent repetition. In experiments on 2,000 graph‑coloring instances, CORE cuts median verifier calls by up to 39.8% compared to chronological repair, and improves success rates on five reasoning tasks, achieving 75.9% with Qwen2.5‑7B‑Instruct and 84.2% with Qwen3‑8B versus 72.5% and 81.8% for Tree of Thoughts. The approach also reduces verifier calls and generated tokens on both language‑model backbones.
By Siyu Song, Rui Xu, Jia Lin, Kai Liu, Weifang Wang
arXiv:2607. 09999v1 Announce Type: cross Abstract: We show that post-training quantization can silently alter how large language models reason even when task accuracy is preserved.
By Renuka Oladri, Mohan Vamsi Varadaraju Priya, Jerry Wu
arXiv:2609.12181v1 Announce Type: new
Abstract: Vision-language and language models increasingly interpret materials data, yet benchmarks report that they hallucinate invalid properties and violate p...
By Hasan Kurban, Rasul Khanbayov, Mustafa Kurban
arXiv:2608. 09190v1 Announce Type: new Abstract: GK is a query-directed first-order prover that extends ordinary resolution-based proof search with explicit positive and negative claims, numerical confidence values, and prioritized default rules with exceptions.
By Tanel Tammet
arXiv:2608.28725v1 Announce Type: new
Abstract: Large language models (LLMs) are increasingly used as graders, verifiers, and process auditors, but most mathematical evaluations still emphasize final...
By Fateme Mazdarani, Carlos Toxtli
The paper investigates in-context binding errors in language models, showing that a linear probe can recover correct entity bindings from frozen hidden states even when the model outputs incorrect bindings. Across 16 checkpoints, the probe’s accuracy on failure cases surpasses a baseline by about 0.196, and a probe‑based score improves failure detection over the model’s confidence by 0.079 AUROC. Steering the residual stream toward the probe‑decoded binding further boosts accuracy by an average of 0.168 across eight models.
By Manas Venkata Sai Ravulapalli, Samrath Singh Chadha, Abhinav M. Hari