arXiv AI

Should SLMs be in the Driver's Seat? A Study of Local Models as Interpreters in Cloud-Written Agent Programs

arXiv AI
Aug 18

From LLM Inference to Agentic Workloads: Characterization and Implications for Serving Systems

arXiv:2608. 15127v1 Announce Type: cross Abstract: Agentic applications are shifting AI serving from isolated model inference to long-running workloads in which LLMs coordinate tools, environments, and persistent state.

By Chaokun Chang, Yukun Zhou, Kaihua Fu, Dakai An, Tianyu Feng, Hanfeng Lu, Sheng Yao, Pu Guo, Yinghao Yu, Yizhou Shan, Bo Li, Binhang Yuan, Wei Wang
arXiv AI
Oct 7

Do I Need the Cloud? Uncertainty-Aware Step-Level Handoff for Small Language Model Agents

The paper introduces STEPGATE, an uncertainty‑aware handoff framework that evaluates each step of a small language model (SLM) agent and selectively escalates difficult steps to a stronger model. On a 52‑task single‑step benchmark, the Qwen2.5‑1.5B/7B pair achieved 82.7% task success with only 30.8% escalation, outperforming local‑only and random escalation baselines. In multi‑turn tests, STEPGATE reached 69.0% trajectory success and 84.0% action success while using only 30.0% cloud actions, demonstrating that step‑level escalation can close much of the performance gap to a stronger backend with fewer remote tokens.

By Abolfazl Younesi
arXiv Computation and Language
5d ago

Agentic World Modeling for Agent Test-time Scaling without Environment Replica

Agentic World Modeling (AWM) redefines next-observation prediction as a multi-turn evidence-acquisition task, allowing agents to query a real environment safely and construct observations without mutating shared state. Integrated with trajectory-level test-time scaling, AWM enables parallel exploration over a single shared environment, outperforming single-shot simulation by 40.07% and trained world-model baselines by 12.36% on benchmarks such as τ-bench, AppWorld, and OfficeBench. This approach provides a practical foundation for safe and scalable agentic test-time scaling in shared, non-resettable environments.

By Wendi Li, Huseyin A. Inan, Wei-Ning Chen, Yanzhi Chen, Chetan Bansal, Saravan Rajmohan, Sharon Li
arXiv AI
Sep 15

OrchSLM: Probing the Dynamics of Small Language Model Orchestration

OrchSLM is a routing framework that unifies non‑interactive orchestration methods for small language models (SLMs). It allows heterogeneous SLMs to independently generate candidate solutions while a router manages their cached outputs without further model interaction. By systematically probing OrchSLM, the study shows how orchestration behavior depends on task structure, model‑pool composition, and multi‑agent consensus.

By Chengxi Zhang, Yu Yao
arXiv Computation and Language
Oct 19

EdgeWise: Bringing Long-Horizon Agents to the Edge via Selective Cloud Delegation

arXiv:2610.17560v2 Announce Type: replace Abstract: Complex and long-horizon agent tasks often rely on frontier models deployed in the cloud to achieve strong performance. However, using these models...

By Zeru Shi, Jindong Jiang, Ryo Hachiuma, Yonggan Fu, Byung-Kwan Lee, Peter Belcak, Saurav Muralidharan, Ayush Agarwal, Ruixiang Tang, Ambrish Dantrey, Pavlo Molchanov, Anjul Patney, Subhashree Radhakrishnan
arXiv Machine Learning
Aug 10

Cascade: Exploiting SLO-Aware latency budget for fair and high goodput LLM inference serving

arXiv:2608. 06557v1 Announce Type: cross Abstract: The reasoning and agentic capabilities of large language models have expanded the range of applications they support, from short interactive exchanges to long, compute-heavy requests.

By Muhammad Adnan, Rohan Mahapatra, Prashant J. Nair, Daniel Berger, Pantea Zardoshti, Rodrigo Fonseca, Esha Choukse
arXiv AI
Jun 2

CoMIC: Collaborative Memory and Insights Circulation for Long-Horizon LLM Agents in Cloud-Edge Systems

arXiv:2606. 00756v1 Announce Type: new Abstract: Deploying lightweight Large Language Model (LLM) agents on edge servers can reduce latency and move agentic services closer to users, but resource-constrained edge models often struggle with long-horizon tasks that require persistent memory, subgoal tracking, and reflection.

By Yannan Wang, Longli Yang, Zhen Liu, Abhishek Kumar, Carsten Maple