arXiv AI By Bingyu Bai, Cheng Zhang, Erhu Feng

Reuse, Rewrite, or Reject: Evidence-Constrained Trace Reuse for Repeated Mobile GUI Control

Read the original on arXiv AI →

The paper introduces a framework that selectively reuses parts of previously executed mobile GUI control traces to reduce LLM inference overhead. It identifies six categories of variation that can affect trace reuse and shows that individual transitions can still be valid when checked against the current task context and UI state. Using graph-text templates, the system achieves roughly 80% fewer LLM tokens and up to 2‑fold speedups on Android and HarmonyOS devices.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Oct 9

DeltaReplay: Task-Relative Memory Reuse for Mobile GUI Agents

DeltaReplay is a step‑level memory reuse framework for mobile GUI agents that decides how to use stored execution trajectories without modifying them. It stores trajectories as paths in a transition graph, separating each action into a task‑independent operation and task‑specific parameters, and compares each recorded step with the new task and current screen to decide whether to follow, adapt, or ignore it. Experiments on AndroidWorld and SPA‑Bench show that DeltaReplay can increase task success rates by up to 10.3 and 25.0 percentage points over a base agent with the same backbone.

By Yudong Bai, Yihong Chen, Quanming Yao, Yaqing Wang
Hugging Face Trending Papers
Oct 8

DeltaReplay: Task-Relative Memory Reuse for Mobile GUI Agents

DeltaReplay introduces a step-level memory reuse framework for mobile GUI agents that selectively reuses parts of stored execution trajectories without modifying them. By representing trajectories as paths in a transition graph and separating actions into task-independent operations and task-specific parameters, the system compares each recorded step with the new task and current screen to decide whether to follow, adapt, or ignore it. Experiments on AndroidWorld and SPA-Bench show that this selective reuse can boost task success rates by up to 10.3 and 25.0 percentage points over a baseline agent.

arXiv AI
Sep 11

JarvisGUI: Towards Cross-Device GUI Agents with Dynamic Task Composition

JarvisGUI is a new benchmark that tests GUI agents on cross-device workflows involving Android, Windows, and Ubuntu, requiring transfer of intermediate results and coordination across heterogeneous platforms. It formulates tasks as input-output transformations under a lightweight type system, enabling automatic composition of multi-step, cross-device workflows and dynamic evaluation within a unified framework. The benchmark reveals that state-of-the-art open-source GUI agents struggle with state-transfer awareness, cross-platform contextual reasoning, and long-horizon dependency management, exposing a critical capability gap invisible to existing benchmarks.

By Zixiang Chen, Yuheng Lu, Zihao Cheng, Zeming Liu, Jizeng Bai, Ziye Huang, Zhiyin Lin, Zihan Li, Yuhang Guo, Yunhong Wang, Haifeng Wang
arXiv AI
Sep 16

EchoPath: Execution-Level Replayable Memory for GUI Agents

EchoPath is a model‑agnostic framework that transforms validated GUI interaction trajectories into standardized, parameter‑controlled memories, enabling agents to replay specific GUI actions deterministically. Each memory records task intent, preconditions, input parameters, GUI evidence, validation provenance, and lifecycle state, and uses an image‑based target‑reaiming algorithm to adjust coordinates for the current screen before execution. Experiments on real computer‑use tasks show that EchoPath cuts median token cost by over 90 % and median execution time by about 60 %.

By Yao Zhao, Aditya Shanmugham, Swastik Roy, Yanxun Xu