arXiv Machine Learning June 2, 2026 By Jianru Ding, Ryien Hosseini, Pouya Mahdi Gholami, Mingyuan Xiang, Henry Hoffmann

Observation, Not Prediction: Conversation-Level Disaggregated Scheduling for Agentic Serving

Read the original on arXiv Machine Learning →

arXiv:2606. 01839v1 Announce Type: cross Abstract: LLM-based agents resolve a user task through many turns of dependent inference and tool calls, producing a workload whose total cost is unknown when the task arrives.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

1d ago

Denoising Surface: Modeling and Predicting Inference Cost for Diffusion LLM Serving

The paper introduces the Denoising Workload Surface (DWS), a two‑dimensional probability surface that captures the block‑autoregressive generation structure of diffusion large language models (dLLMs). By preserving both output block and within‑block denoising step information, DWS enables a lightweight, prompt‑only predictor to estimate per‑request inference cost accurately, even on a single CPU core. In real‑world serving experiments, DWS reduces cost‑prediction error by up to 2.5× and improves end‑to‑end latency for online chatbots by up to 1.92×.

By Haoyu Zheng, Fangcheng Fu, Binhang Yuan, Yongqiang Zhang, Liang Deng, Hao Wang, Yuanyuan Zhu, Xiao Yan, Jiawei Jiang

llms diffusion

arXiv AI

Jul 3

Towards Load-Aware Prefill Deflection for Disaggregated LLM Serving

arXiv:2607. 02043v1 Announce Type: cross Abstract: Disaggregated LLM serving runs prefill and decode on separate GPU pools to keep the two phases from interfering.

By Shrikara Arun, Anjaly Parayil, Srikant Bharadwaj, Renee St. Amant, Victor R\"uhle

llms benchmarks

Hugging Face Trending Papers

Jul 5

Sangam: Efficiently Serving Diffusion LLMs with the AR Stack

Diffusion language models (dLLMs) generate text by iteratively denoising a masked response and can commit multiple output positions per model invocation. Their bidirectional attention prevents exact autoregressive-style KV caching, since committing one position shifts the KV activations of all others.

llms diffusion

arXiv Machine Learning

Jul 7

Sangam: Efficiently Serving Diffusion LLMs with the AR Stack

arXiv:2607. 04206v1 Announce Type: cross Abstract: Diffusion language models (dLLMs) generate text by iteratively denoising a masked response and can commit multiple output positions per model invocation.

By Nitin Kedia, Saurabh Agarwal, Myungjin Lee, Aditya Akella

llms diffusion

arXiv AI

Aug 18

Decode-Branch Transformers: Decoupling the Primary Prefill Path from Additional Decode Computation

arXiv:2608. 12385v2 Announce Type: replace Abstract: As large language models serve ever more requests, cumulative inference cost is growing relative to the one-time cost of training.

By Liming Liu, Mingze Wang, Tuo Zhao

llms rag efficiency

arXiv Machine Learning

Jun 16

MARS: Efficient, Adaptive Co-Scheduling for Heterogeneous Agentic Systems

arXiv:2604. 26963v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly deployed as the execution core of autonomous agents rather than as standalone text generators.

By Yifei Wang, Hancheng Ye, Yechen Xu, Cong Guo, Chiyue Wei, Qinsi Wang, Dongting Li, Tingjun Chen, Hai "Helen" Li, Danyang Zhuo, Yiran Chen

llms agents efficiency

One email a morning, machine-written

Related stories

Denoising Surface: Modeling and Predicting Inference Cost for Diffusion LLM Serving

Towards Load-Aware Prefill Deflection for Disaggregated LLM Serving

Sangam: Efficiently Serving Diffusion LLMs with the AR Stack

Sangam: Efficiently Serving Diffusion LLMs with the AR Stack

Decode-Branch Transformers: Decoupling the Primary Prefill Path from Additional Decode Computation

MARS: Efficient, Adaptive Co-Scheduling for Heterogeneous Agentic Systems