The paper investigates whether large language models’ reasoning traces truly contain early, informative signals or merely reflect budget and difficulty confounds. Using a restart‑controlled truncation probe, the authors compare continuation success rates against from‑scratch restart curves across 178 problem‑model pairs, finding that only one case shows prefix‑limited success and that continuing a model’s own prefix generally outperforms restarting. A difficulty‑controlled test and two generation‑free analyses reveal that early internal signals do not carry outcome information beyond a problem‑difficulty baseline, underscoring the need for proper counterfactual controls.
By Yigit Utku Bulut
The study investigates how language models equipped with tools can still produce unsupported final claims, even when a single tool call could resolve the uncertainty. It defines two metrics—occurrence (how often unsupported claims arise) and conditional repair (how often they are fixed when evidence is provided). Experiments on Qwen3-32B and Gemma 4 show that providing the missing evidence consistently repairs all unsupported claims in the Qwen3-32B setup, while the Gemma 4 model never produced unsupported claims under the tested conditions.
By Justin Bronder
arXiv:2607. 16646v1 Announce Type: cross Abstract: Large language models now translate natural-language descriptions of decision problems into solver-ready optimization models, but they fail silently.
By Haifeng Li, Mo Hai
The paper demonstrates that language models often accept and use unsupported premises when answering questions, even though the source text does not provide the necessary information. By manipulating whether an unsupported fact is usable, the authors show that models’ refusal rates drop significantly while their commitment to an answer increases. This behavior is consistent across multiple models and datasets, indicating that models’ verification checks are heavily influenced by the presence of usable material rather than strict source support.
By Naveen Madiraju
arXiv:2607. 20492v1 Announce Type: cross Abstract: Language models in production do not write prose.
By Rana Muhammad Usman
arXiv:2610.33949v1 Announce Type: cross
Abstract: Pipelines that require structured outputs from hosted language models often specify a Pydantic schema in the prompt, validate the response, and retry...
By Amit Verma
arXiv:2606. 09410v1 Announce Type: new Abstract: Prior work treats structured output as a reasoning tax, but this framing is incomplete: the cost of formatting depends strongly on a model's spare capacity.
By Hengxin Fan
arXiv:2609.14758v1 Announce Type: cross
Abstract: Tool-augmented language models are evaluated on whether they reach the right answer, not on whether they report honestly when a tool fails to supply...
By Arham Sethi, Arsen Kenzhebayev, Saanvi Paturi, Vatsal Raina, Vyas Raina, Ivaxi Sheth
arXiv:2606. 09046v1 Announce Type: new Abstract: Useful audits reveal not only how often a model fails, but also where its failures concentrate.
By Vyzantinos Repantis, Ameya Gawde, Harshvardhan Singh
Benchmark gains are often treated as evidence of greater LLM capability. Yet the same gain can reflect different changes in model behavior.
The paper investigates using Large Language Models (LLMs) as calibration agents for expensive engineering simulators by conditioning them on physics-labeled prompts that describe simulation errors. In tests on a cylinder-wake Large-Eddy Simulation and a damped-oscillator inverse problem, physics-informed LLM proposals outperform conventional derivative-free optimizers in early trials, but the advantage fades after fifteen evaluations. A hybrid workflow that uses the LLM as a warm-start followed by a conventional optimizer achieves target accuracy in only nine simulator runs, outperforming both the LLM alone and standard optimizers.
By Ruo-Qian Wang
ToolGate is an executable acceptance pipeline designed to streamline the creation of scientific benchmarks that rely on specialist software. It evaluates each model-generated item through three gates: (1) an executable solution script must reproduce the proposed answer, (2) a randomized no‑tool screen rejects items solvable without the software, and (3) a tool‑using agent must solve the item within a time limit. In a FEniCSx instantiation, 500 generation attempts produced 128 unique, verified benchmark items after successive filtering.
By Ke Zhang, Yankang Liu, Roya Zandi, Maziar Raissi