The paper investigates how increasing inference-time computation—via wider beam search or sample‑plus‑vote—affects performance on grammar‑constrained text‑to‑SQL tasks for small language models. Using the Qwen2.5‑Instruct family (0.5B–7B parameters) on the Spider benchmark, the authors find that larger models consistently outperform higher inference compute on the same model size, and that beam search yields better accuracy than sample‑plus‑vote under matched budgets. These results suggest that, unlike unconstrained settings, scaling inference compute does not compensate for smaller model size when strict grammar constraints are applied.
By Ty Chermsirivatana, John MacCormick
Large language models can solve complex multi‑hop tasks but often fail on simple two‑hop queries, even when each hop is individually correct. In a controlled symbolic setting, the authors find that models generalize reliably when the second hop follows the training distribution, but always fail when it deviates. Mechanistic analysis shows that successful generalization relies on consistent intermediate representations across contexts, whereas failures arise from a mismatch between lower‑layer representation construction and upper‑layer mapping to outputs. The study proposes a recurrent‑style training strategy that improves out‑of‑distribution two‑hop generalization.
By Zili Zhang, Yilin Wang, Heng Wang, Herun Wan, Minnan Luo
CacheSpec is an inference optimization framework that transforms Program-of-Thoughts (PoT) style programs into reusable cache objects for large language models. By employing a small model for semantic variable extraction on cache hits and speculative drafting during target-LLM generation, CacheSpec reduces inference latency and improves cache reuse. Experiments on shopping, web, formula, and code QA datasets demonstrate up to 3.1× speedup in latency and 2.8× throughput gains over traditional PoT methods, while maintaining or improving task quality.
By Jingquan Chen, Jie Feng, Jinghua Piao, Shaogang Hu, Yong Li
arXiv:2512. 20638v2 Announce Type: replace-cross Abstract: The evaluation of large language models relies heavily on standardized benchmarks.
By Maty Bohacek, Nino Scherrer, Nicholas Dufour, Thomas Leung, Christoph Bregler, Stephanie C. Y. Chan
arXiv:2602.21061v2 Announce Type: replace
Abstract: Many current paths to more advanced AI depend on the assumption that large language models (LLMs) can generalize learned relationships to solve com...
By David Koplow, Tomer Galanti, Tomaso Poggio
arXiv:2607. 08399v1 Announce Type: cross Abstract: Large language models process prompts by propagating activations through dozens of layers before generating a response.
By Thibaud Ardoin, Semira Einsele, Evis Bregu, Gerhard Wunder
arXiv:2605. 00994v2 Announce Type: replace-cross Abstract: Finetuning can significantly modify the behavior of large language models, including introducing harmful or unsafe behaviors.
By Mohammed Abu Baker, Luca Baroni, Dan Wilhelm
arXiv:2602.00377v3 Announce Type: replace
Abstract: Existing knowledge probing methods rely on pre-defined queries, limiting extraction to known concepts. We introduce DecompressionLM, a stateless fr...
By Zhaochen Hong, Jiaxuan You
arXiv:2606. 24026v1 Announce Type: new Abstract: Mechanistic interpretability has made substantial progress in automatically localizing circuits, but explaining what localized components do remains labor-intensive and difficult to standardize.
By Ayan Antik Khan, Harsh Kohli, Yuekun Yao, Huan Sun, Ziyu Yao
arXiv:2609.15992v1 Announce Type: new
Abstract: Recent advances in large language models (LLMs) have rendered them necessary for Natural Language Processing (NLP) tasks, and their high inference cost...
By Foivos Charalampakos, Md Ibrahim Ibne Alam, Iordanis Koutsopoulos, Koushik Kar
arXiv:2608. 13926v1 Announce Type: new Abstract: Large language models have made natural language interfaces to databases (NLIDB) newly credible, but LLM text-to-SQL systems fail in a way that matters for deployment: a hallucinated column or a mis-aggregated total yields a fluent wrong answer, indistinguishable at the point of use from a right one.
By Zhelun (Allen), Wu
Large language models have made natural language interfaces to databases (NLIDB) newly credible, but LLM text-to-SQL systems fail in a way that matters for deployment: a hallucinated column or a mis-a...