arXiv:2609.13149v1 Announce Type: new
Abstract: For local large language model agents, active context is a scarce resource: memory capacity, prefill latency, cache growth, and service objectives all...
By Aditya Karnam Gururaj Rao, Arjun Jaggi
arXiv:2610.07094v1 Announce Type: cross
Abstract: LLM deployment is shifting from single-turn completion to agentic trajectories in which a model plans, calls tools, reads results and reasons at test...
By Abbas Raza Ali, Muhammad Ajmal Siddiqui, Moona Zahid
The paper introduces a Mixture-of-Agents (MoA) approach to quantify the per-token compute required by large language models. By having a panel of fifteen models of varying sizes attempt to reproduce each token, the authors define the smallest successful agent’s inference cost as the token’s sufficient compute, providing an upper bound on necessary computation. Experiments on benchmarks show that a 0.5B model can reproduce most tokens, and that the MoA-derived compute map can reduce latency in model routing and drafting tasks while improving or maintaining accuracy.
By Zhixu Du, Weijia Han, Hai Helen Li, Yiran Chen
arXiv:2609.35760v2 Announce Type: replace-cross
Abstract: When a large language model (LLM) agent executes the same task, token consumption can vary by over an order of magnitude across runs. The age...
By Chaoqian Ouyang, Ling Yue, Libin Zheng, Hanghui Guo, Shengxiang Xu, YiShu Wang, Ran Li, Jian Yin, Shaowu Pan, Shimin Di
A transformer language model performs a bounded amount of computation per token, and recent work by Vishal Sikka, former CEO of Infosys, argues that this bound limits which tasks a model can carry out...
arXiv:2607. 13034v1 Announce Type: new Abstract: Large language model (LLM) agents increasingly automate multi-step engineering and informatics workflows, yet they rarely ask how much effort a task actually requires.
By Junjie Yin, Xinyu Feng
arXiv:2609.22114v1 Announce Type: new
Abstract: Context compression is widely proposed as a way to cut the token bill of LLM coding agents, and public benchmarks report that aggressive compression pr...
By Luzhuo Chen, Jiayu Shi
arXiv:2607. 08032v1 Announce Type: new Abstract: Large language models, and the agents built on them, spend an ever-growing share of their compute and memory on remembering: caching attention keys and values, carrying long prompts, maintaining recurrent state, and storing what happened in previous turns and sessions.
By Ashwin Gerard Colaco, Nada Lahjouji
Nexus introduces a depth‑adaptive KV‑cache splicing and retrieval‑decoupled tool routing mechanism for agentic large language models that reduces the time‑to‑first‑token (TTFT) by decoupling tool routing from the expensive schema re‑encoding step. It uses an INT8 semantic lookaside buffer to select tools via retrieval and generates arguments from a compressed textual signature, maintaining about 89% routing accuracy even as the tool registry scales to 250 tools. Additionally, Nexus can splice compiled schema KV blocks into the live context, repairing the seam with a depth‑adaptive suffix redecode when rotary position embedding drift exceeds a threshold, ensuring output fidelity while achieving up to 1.7× TTFT speedup at moderate depth.
By Mustafa Arslan
The paper introduces Galahad, a memory layer that stores a transformer language model’s key‑value state for blocks of text, allowing subsequent requests to reuse previously computed attention rather than recomputing it. On seven real‑world datasets, 98.7% of prompt tokens were already read, and with Galahad the model could attend to an entire 97,000‑token corpus, achieving 98–100% recall on a 100‑fact test while reducing inference time and energy consumption dramatically. The approach was validated across 30 models and all runtimes, demonstrating that stateful inference can replace stateless serving without loss of accuracy.
By Sietse Schelpe
The paper introduces InFlowOp, a label‑free optimization framework that assigns costs to each decision in a multi‑agent workflow, balancing agent competence against execution time. It determines task granularity and agent assignment before execution and corrects faults during execution using the same cost metric. The authors also present Braid, a benchmark for multi‑agent coordination, and show that InFlowOp outperforms single‑agent baselines by up to 11.97% across various domains.
By Xuehang Guo, Haoyu Wang, Shengyu Chen, Zach Chen, Wei Cheng, Qingyun Wang, Haifeng Chen
arXiv:2606. 29914v1 Announce Type: cross Abstract: Agent memory systems are increasingly evaluated against RAG and full-context baselines, but reported gains often mix changes in the memory method with changes in the language model, embedding model, or retrieval pipeline, making it unclear what is actually being measured.
By Kuan Wang