The paper introduces a Mixture-of-Agents (MoA) approach to quantify the per-token compute required by large language models. By having a panel of fifteen models of varying sizes attempt to reproduce each token, the authors define the smallest successful agent’s inference cost as the token’s sufficient compute, providing an upper bound on necessary computation. Experiments on benchmarks show that a 0.5B model can reproduce most tokens, and that the MoA-derived compute map can reduce latency in model routing and drafting tasks while improving or maintaining accuracy.
By Zhixu Du, Weijia Han, Hai Helen Li, Yiran Chen
The paper argues that large language models need adaptive reasoning rather than fixed reasoning budgets. It shows that over‑reasoning leads to high computational cost without accuracy gains, while under‑reasoning results in incorrect or incomplete solutions. The authors evaluate these failure modes on MATH‑500 and the GAIA benchmark, highlighting the need for dynamic reasoning allocation in agentic AI systems.
By Md Jueal Mia, M. Hadi Amini
arXiv:2609.35760v2 Announce Type: replace-cross
Abstract: When a large language model (LLM) agent executes the same task, token consumption can vary by over an order of magnitude across runs. The age...
By Chaoqian Ouyang, Ling Yue, Libin Zheng, Hanghui Guo, Shengxiang Xu, YiShu Wang, Ran Li, Jian Yin, Shaowu Pan, Shimin Di
arXiv:2608. 13571v1 Announce Type: cross Abstract: When a language model fails to answer a query on the first attempt, an agentic system retries, consuming additional tokens each time.
By Heming Fu, Shan Lin, Qianqian Xie, Guojun Xiong
arXiv:2607. 06413v1 Announce Type: cross Abstract: Large language model coding agents increasingly perform open-ended data modeling and analysis.
By Hao He, Xueying Liu, Chris J. Kuhlman, Xinwei Deng
The paper investigates how large‑language‑model (LLM) based AI agents mix latency, local resource usage, and container bottlenecks when processing user requests that involve remote LLM calls and local tool execution. By measuring three representative tasks—retrieval‑augmented question answering, web search, and software coding—the authors show that agents exhibit diverse resource dynamics, with concurrent requests revealing task‑specific bottlenecks in CPU, disk I/O, and memory. Leveraging these insights, they propose CPU‑aware tool admission and task‑aware CPU allocation, achieving up to a 5.4× speed‑up for CPU‑sensitive tasks and a 32% reduction in average latency across multiple tasks.
By Wonmi Choi, Minuk Park, Zhixiong Niu, Yongqiang Xiong, Chuck Yoo, Gyeongsik Yang