The paper introduces Galahad, a memory layer that stores a transformer language model’s key‑value state for blocks of text, allowing subsequent requests to reuse previously computed attention rather than recomputing it. On seven real‑world datasets, 98.7% of prompt tokens were already read, and with Galahad the model could attend to an entire 97,000‑token corpus, achieving 98–100% recall on a 100‑fact test while reducing inference time and energy consumption dramatically. The approach was validated across 30 models and all runtimes, demonstrating that stateful inference can replace stateless serving without loss of accuracy.
By Sietse Schelpe
A transformer language model performs a bounded amount of computation per token, and recent work by Vishal Sikka, former CEO of Infosys, argues that this bound limits which tasks a model can carry out...
arXiv:2610. 29052v1 Announce Type: cross Abstract: A typed decision maps evidence, a question and a fixed set of options to a probability for every option, computed in one forward pass, by reading a language model's logits or a small head over its hidden states, instead of generating and parsing text.
By Yuchuan Weng, Chi Liu, Xin Chen, Srinivasan Manoharan
arXiv:2609.38006v1 Announce Type: new
Abstract: Running a language model locally offers advantages in privacy, latency, and cost, but local hardware fits only small models, which are less capable tha...
By Kenan Alkiek, Moontae Lee, David Jurgens, V. G. Vinod Vydiswaran
The paper evaluates a manager‑worker scaffold that uses a shared filesystem workspace to orchestrate multi‑agent large language model (LLM) coding tasks without training or tuning. Across nine models—including five open‑weight and four closed‑weight systems—the scaffold yields statistically significant accuracy gains for some models (e.g., Qwen3.8‑27B, GPT‑5.6‑Luna, GPT‑5.6‑Terra, Kimi‑K3, Minimax‑M3) while producing null or negative effects for others (e.g., Qwen3.6‑35B). The study shows that the manager can triple token usage but still achieves higher accuracy at a fraction of the cost compared to larger single‑pass models, with key mechanisms identified as context management and problem decomposition.
By Victor Gao (Sang Won), Vida Khosrowshahi (Sang Won), Ali Khosrowshahi (Sang Won), Xihao Sun (Sang Won), Juhyun Lee (Sang Won), Simon (Sang Won), Lee
Verdict-MQ is an open encoder that processes a single piece of state once and then answers multiple typed questions about that state, avoiding separate passes for each question. By using a structured attention mask, it ensures the state encoding is independent of question order and prevents questions from influencing one another. On the typed‑decisions benchmark, Verdict‑MQ outperforms a larger specialist model, achieving higher accuracy and better calibration, while on other benchmarks it shows mixed results compared to fine‑tuned baselines.
By Avdhesh Singh Chouhan, Jack Perry
LEV is a decision model that performs reasoning by looping its layers instead of generating text, allowing it to deepen its computation on a per-question basis. Trained jointly across all loop steps, a single LEV checkpoint can answer at any depth, improving accuracy—especially on hard questions—while keeping inference fast. On benchmark tests, LEV outperforms dense and cascade models by 5.4 to 8.8 accuracy points with sub‑50 ms latency, and on‑demand depth selection saves 15–24% of computation for minimal accuracy loss.
By Dingjie Song, Zhiling Yan, Yuxuan Zhang, Lichao Sun
The paper introduces a coroutine-bridge harness that lets a language model emit a Python program to manage tool calls in the CAR-bench evaluation. By decoupling model invocations from tool round-trips, the approach reduces model calls to a median of two per task while maintaining seven agent turns, achieving a median latency of 1.8 s on a Cerebras gpt‑oss‑120b. The harness achieved 60.0 % Pass³ on the official hidden evaluation, outperforming the baseline by 4.5× and matching frontier-model agents on GPT‑5.5, all while keeping the prompt largely cached and minimizing input compute.
By Ivan Matveev
arXiv:2609.25686v1 Announce Type: cross
Abstract: Long-horizon assigned work requires an LLM agent to track the state of a task: which steps are done, blocked, cancelled, or open to repetition. Agent...
By Chenyu Zhang, Wonbin Kweon, Jiawei Han
arXiv:2608. 15810v1 Announce Type: new Abstract: Runtime compression of serving state trades quality for capacity with no priced guarantee: systems adapt precision on load signals with no soundness statement, and certified approaches budget request-level risk by a union bound over a pre-declared event count.
By Fanzhe Wei, Li Liu
The paper reports on training and calibration experiments for three open models on the ARC‑AGI‑2 benchmark, using offline runs on four L4 GPUs for up to 12 hours. For the NVARC Qwen3‑4B grid model, a curated corpus of 12,000 executable rules and 95,993 tasks was built, and fine‑tuning on 1,920 real task records raised the semi‑private score from 28.96 to 33.06, with no synthetic‑task checkpoint outperforming the real‑task model. For the Qwen3.8‑27B reasoning model, fine‑tuning on 1,036 of its own reasoning traces and a specialized inference procedure increased the score from 26.81 to 44.03, while the Qwen3.8‑Flash‑Next mixture‑of‑experts model achieved a best score of 56.67 after expert pruning, routing statistics adaptation, and an FP8 indexer cache.
By Nguyen Minh Thang, Trinh Thi Phuong
Large language model (LLM) agents now fix software, operate computers and serve customers, but their harnesses, the programs that run them, call a large LLM at every step, which makes long tasks slow...