arXiv:2608. 07528v1 Announce Type: new Abstract: Linear probes detect corrupted context in language models with near-perfect accuracy, yet this does not translate into reliable failure prediction.
By Jyotin Goel, Ipshita Bandyopadhyay, Justin Shenk
arXiv:2606. 05145v1 Announce Type: cross Abstract: When post-trained language models fail on reasoning problems, the common test-time-scaling response is to spend more compute on additional attempts, and the failed traces play no further role.
By Nizar Islah, Istabrak Abbes, Irina Rish, Sarath Chandar, Eilif B. Muller
A 0.6B language model consistently answers YES to 1,200 logical tests, yet its behavior shows no discrimination. Linear probes reveal the correct verdict with high AUC (0.96) and transfer to unseen structures, but a single scalar readout fails due to a saturated decision threshold offset by +4.6 σ. Adjusting this threshold restores behavior accuracy from 50 % to 81 % and improves higher‑scale models, demonstrating that miscalibrated readouts, not hidden knowledge loss, drive performance gaps.
By Gnaneswar Villuri, Hashmath Shaik, Alex Doboli
The paper investigates how latent reasoning in language models is accessed by probing, focusing on a digit copied from the question in two models, Ouro-1.4B and Qwen2.5-1.5B. By comparing prompts that differ only in that digit, the authors show the digit is present in the final prompt token with a shared code readout of about 0.9, yet a standard single‑prompt probe reads it only weakly (0.36 in Ouro and 0.19 in Qwen). The study introduces a protocol with a synthetic null to test computed values against copies and mixtures of inputs, revealing that while the digit is readable, it is not used in subsequent reasoning steps.
By Aur Shalev Merin
The study investigates how language models equipped with tools can still produce unsupported final claims, even when a single tool call could resolve the uncertainty. It defines two metrics—occurrence (how often unsupported claims arise) and conditional repair (how often they are fixed when evidence is provided). Experiments on Qwen3-32B and Gemma 4 show that providing the missing evidence consistently repairs all unsupported claims in the Qwen3-32B setup, while the Gemma 4 model never produced unsupported claims under the tested conditions.
By Justin Bronder
The paper investigates whether large language models’ reasoning traces truly contain early, informative signals or merely reflect budget and difficulty confounds. Using a restart‑controlled truncation probe, the authors compare continuation success rates against from‑scratch restart curves across 178 problem‑model pairs, finding that only one case shows prefix‑limited success and that continuing a model’s own prefix generally outperforms restarting. A difficulty‑controlled test and two generation‑free analyses reveal that early internal signals do not carry outcome information beyond a problem‑difficulty baseline, underscoring the need for proper counterfactual controls.
By Yigit Utku Bulut