arXiv Machine Learning August 4, 2026 By Hao Mark Chen, Jinnan Guo, Wayne Luk, Hongxiang Fan

AOSpec: Action and Observation Co-Speculation for Low-Latency Agent Serving

Read the original on arXiv Machine Learning →

arXiv:2608. 00881v1 Announce Type: new Abstract: Large language model agents increasingly act through stateful tools, yet model generation and environment execution remain serialized at every step.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Jul 7

SPORK: Self-Speculative Forking to Accelerate Agentic LLM Inference

arXiv:2607. 03333v1 Announce Type: cross Abstract: LLM agents are becoming a common interface for research, coding, and question answering, yet their Thought-Action-Observation loop is often serial: the model reasons, emits a tool call, then idles the GPU until the result returns.

By Huajun Bai, Weiwei Lv, Huichuan Zheng, Youyou Lu, Jiwu Shu

llms agents nlp benchmarks

arXiv Computation and Language

3d ago

TomasuLLM: Out-of-Order Speculative Execution for LLM Agents

arXiv:2609.38201v1 Announce Type: new Abstract: Long-running tools can dominate coding-agent latency: compilers, test suites, and repository commands take seconds to minutes while the agent idles. Th...

By Jiangnan Yu, Ceyu Xu, Mengming Li, Shiyu Huang, Yiran Xia, Jian Weng, Hui Xue, Haohui Mai, Yuan Xie

llms agents benchmarks

arXiv AI

Aug 18

From LLM Inference to Agentic Workloads: Characterization and Implications for Serving Systems

arXiv:2608. 15127v1 Announce Type: cross Abstract: Agentic applications are shifting AI serving from isolated model inference to long-running workloads in which LLMs coordinate tools, environments, and persistent state.

By Chaokun Chang, Yukun Zhou, Kaihua Fu, Dakai An, Tianyu Feng, Hanfeng Lu, Sheng Yao, Pu Guo, Yinghao Yu, Yizhou Shan, Bo Li, Binhang Yuan, Wei Wang

llms agents benchmarks

arXiv Machine Learning

Jul 31

Why Are GUI Agents Correct but Late? Decode on the Decision-Time Critical Path, Tested with Pre-Compiled Policy Trees

arXiv:2607. 28399v1 Announce Type: new Abstract: Computer-use agents often fail on transient GUI events because they produce the correct action only after the relevant window has already closed.

By Zihan Dong, Rui Qian, Qishi Zhan, Dongshen Peng, Kaixin Li, Yu Li

agents multimodal benchmarks

arXiv AI

Sep 4

Speculative Macro Commit for Faster Tool-Using Agents

Speculative Macro Commit (SMC) is a runtime technique for tool‑using language‑model agents that separates an authoritative actor model from a faster speculative drafter model. The drafter predicts and executes future action chains on a snapshot, storing recurring multi‑action patterns in a macro library. When the actor’s next tool call aligns with a drafted action, SMC commits the pre‑executed steps, reducing latency by up to 18.59% on certain benchmarks while maintaining accuracy.

By Zeyu Liu, Souvik Kundu, Peter A. Beerel

llms agents

arXiv AI

Jun 17

PreAct: Computer-Using Agents that Get Faster on Repeated Tasks

arXiv:2606. 17929v1 Announce Type: new Abstract: Computer-using agents drive real software through the screen -- clicking and typing -- but they solve every task from scratch: asked to repeat a task, an agent re-reads the screen, re-reasons every tap, and pays the full cost again.

By Bojie Li

llms rag agents benchmarks safety

One email a morning, machine-written

Related stories

SPORK: Self-Speculative Forking to Accelerate Agentic LLM Inference

TomasuLLM: Out-of-Order Speculative Execution for LLM Agents

From LLM Inference to Agentic Workloads: Characterization and Implications for Serving Systems

Why Are GUI Agents Correct but Late? Decode on the Decision-Time Critical Path, Tested with Pre-Compiled Policy Trees

Speculative Macro Commit for Faster Tool-Using Agents

PreAct: Computer-Using Agents that Get Faster on Repeated Tasks