Hugging Face Trending Papers

Speculative Macro Commit for Faster Tool-Using Agents

Speculative Macro Commit (SMC) is a runtime technique that speeds up tool‑using language‑model agents by having a fast speculative drafter model predict and execute future action chains on a separate environment snapshot. The drafter’s predictions are matched against a macro library of recurring multi‑action skeletons; when the authoritative actor’s next tool call aligns with the first drafted action, SMC commits the remaining pre‑executed steps to the official trajectory. Experiments with Qwen3.5 models show that SMC maintains overall accuracy while cutting latency by up to 18.6% on telecom benchmarks and 44.9% on AppWorld compared to sequential execution.

arXiv AI

Sep 4

Speculative Macro Commit for Faster Tool-Using Agents

Speculative Macro Commit (SMC) is a runtime technique for tool‑using language‑model agents that separates an authoritative actor model from a faster speculative drafter model. The drafter predicts and executes future action chains on a snapshot, storing recurring multi‑action patterns in a macro library. When the actor’s next tool call aligns with a drafted action, SMC commits the pre‑executed steps, reducing latency by up to 18.59% on certain benchmarks while maintaining accuracy.

By Zeyu Liu, Souvik Kundu, Peter A. Beerel

llms agents

arXiv AI

Jul 7

SPORK: Self-Speculative Forking to Accelerate Agentic LLM Inference

arXiv:2607. 03333v1 Announce Type: cross Abstract: LLM agents are becoming a common interface for research, coding, and question answering, yet their Thought-Action-Observation loop is often serial: the model reasons, emits a tool call, then idles the GPU until the result returns.

By Huajun Bai, Weiwei Lv, Huichuan Zheng, Youyou Lu, Jiwu Shu

llms agents nlp benchmarks

arXiv Computation and Language

1d ago

TomasuLLM: Out-of-Order Speculative Execution for LLM Agents

arXiv:2609.38201v1 Announce Type: new Abstract: Long-running tools can dominate coding-agent latency: compilers, test suites, and repository commands take seconds to minutes while the agent idles. Th...

By Jiangnan Yu, Ceyu Xu, Mengming Li, Shiyu Huang, Yiran Xia, Jian Weng, Hui Xue, Haohui Mai, Yuan Xie

llms agents benchmarks

arXiv AI

Jul 29

Speculate While You Reason: Teaching Agents to Predict Their Next Tool Call via Joint Agent-Speculator RL

arXiv:2607. 25816v1 Announce Type: new Abstract: Large language model agents often spend substantial wall-clock time waiting for tool call results.

By Jiabao Ji, Yujian Liu, Li An, Rohit Jain, Gungor Polatkan, Siyu Zhu, Shiyu Chang

llms agents reinforcement-learning efficiency

arXiv Machine Learning

Jul 15

Speculate with Memory: Lossless Acceleration for LLM Agents

arXiv:2607. 12236v1 Announce Type: new Abstract: Speculative execution accelerates LLM agents by using a smaller, cheaper model to predict and pre-launch the next step while the environment is idle.

By Yu Li, Qinyuan Ye, Prafulla Kumar Choubey, Jiaxin Zhang, Chien-Sheng Wu

llms agents benchmarks

arXiv AI

Jun 9

Cost-Aware Speculative Execution for LLM-Agent Workflows: An Integrated Five-Dimension Method

arXiv:2606. 07846v1 Announce Type: cross Abstract: LLM-agent workflows chain model calls and tool invocations, and spend most of their wall-clock time waiting on upstream operations before downstream ones can start.

By Faisal Fareed

llms agents

arXiv Machine Learning

Aug 4

AOSpec: Action and Observation Co-Speculation for Low-Latency Agent Serving

arXiv:2608. 00881v1 Announce Type: new Abstract: Large language model agents increasingly act through stateful tools, yet model generation and environment execution remain serialized at every step.

By Hao Mark Chen, Jinnan Guo, Wayne Luk, Hongxiang Fan

llms agents

arXiv AI

Sep 3

CivBench: A Long-Horizon Benchmark for Tool-Mediated Agents in Civilization VI

CivBench is an open‑source benchmark that evaluates language‑model agents in the long‑horizon, tool‑mediated game Civilization VI using the Model Context Protocol (MCP). Each episode lasts over 300 turns, generating thousands of tool calls across a 76‑tool action space, and includes a narration layer that translates visual game state into structured text. The study characterises agent behaviour across four model families, introducing Proactive Monitoring Rate (PMR) and RAG@10 as interface‑level metrics, and finds that agents often under‑monitor strategic state and fail to execute near‑term commitments despite tool access and explicit guidance.

By Austin Tudor David Andrews, Liam Wilkinson, Jamie Heagerty, Harry Coppock, Jakob Nicolaus Foerster, Rui Ponte Costa

llms rag agents benchmarks

arXiv AI

Jul 28

SpecBox: Speculative Sandbox Scheduling for Efficient LLM Agent Serving

arXiv:2607. 23933v1 Announce Type: cross Abstract: As LLM agents increasingly rely on the Model Context Protocol (MCP) to invoke isolated external sandboxes, disaggregated sandbox deployment introduces a fundamental tension between resource utilization and interactive tail latency.

By Yihui Zhang (Beihang University), Tianyu Wo (Beihang University), Jinghao Wang (Beihang University), Xiaoyang Sun (University of Leeds), Menghao Zhang (Beihang University), Cangzhou Yuan (Beihang University), Li Li (Beihang University), Chunming Hu (Beihang University), Albert Y. Zomaya (The University of Sydney), Renyu Yang (Beihang University)

llms rag agents

arXiv AI

Aug 18

AstronOS: A Unified Execution Model and Runtime for Long-Horizon Agentic Systems

arXiv:2608. 16381v1 Announce Type: new Abstract: Agentic systems often organize execution and state around a single conversation, model invocation, or agent instance, even when real work spans many calls and stages.

By Zhenhang Nie (iFLYTEK Co., Ltd., Hefei, China), Gui Zheng (iFLYTEK Co., Ltd., Hefei, China), Xudong Sun (iFLYTEK Co., Ltd., Hefei, China), Tailong Zhu (iFLYTEK Co., Ltd., Hefei, China), Bin Zhang (iFLYTEK Co., Ltd., Hefei, China)

agents benchmarks

arXiv Machine Learning

Jun 18

EfficientRollout: System-Aware Self-Speculative Decoding for RL Rollouts

arXiv:2606. 18967v1 Announce Type: new Abstract: Reinforcement learning (RL) has become a representative post-training paradigm for LLMs, enabling strong reasoning and agentic capabilities.

By Minseo Kim, Minjae Lee, Seunghyuk Oh, Kevin Galim, Donghoon Kim, Coleman Hooper, Harman Singh, Amir Gholami, Hyung Il Koo, Wonjun Kang

llms agents reinforcement-learning efficiency

arXiv AI

1d ago

Evaluating Agents Across Runtime Contracts: When Mismatch Costs Efficiency or Quality

arXiv:2603.01209v3 Announce Type: replace Abstract: In CodeAct, language-model agents write Python that calls tools and use execution feedback to choose actions. Persistent runtimes preserve Python v...

By Victor May, Van Khue Nguyen, Aaditya Salgarkar, Yishan Wang, Diganta Misra, Huu Nguyen

agents fine-tuning

Related stories