arXiv AI By Xiaolong Li, Xiaohan Xu, Jinyang Li, Xinnuo Xu, Ge Qu, Nan Huo, Jack Williams, Reynold Cheng

When Interfaces Speak: Data-Aware Generative UI Harness for Active Interaction

Read the original on arXiv AI →

The paper introduces GenUI‑Harness, a multi‑agent system that pairs a Tool Agent for data retrieval with a GUI Coder Agent that generates structured interfaces to aid human‑agent interaction. It tackles training challenges by using Dynamic UX for lightweight reward collection and Reward Auditor to monitor reward quality. The authors present UI‑TAU Bench, a benchmark of 1,300 database‑backed tasks, and show that GenUI‑Harness improves Pass@3 scores and reduces dialogue rounds compared to existing models.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Sep 30

AnyAct: Universal Action for Self-Evolving Agents

AnyAct introduces a universal action layer that consolidates diverse tool capabilities into a self‑evolving action space for AI agents operating in open‑world environments. It tackles the scale dilemma, tool non‑stationarity, and heterogeneous feedback by using hierarchical progressive retrieval and test‑time reliability evolution, while a heterogeneous observation grounding module unifies multi‑modal feedback. Evaluations on LiveMCPBench and the newly created OSMCP benchmark show state‑of‑the‑art performance, with significant gains in task success rate and reduced execution steps, especially for models with limited native capabilities.

By Lingrui Xu, Yangqin Jiang, Jiachang Zhang, Xubin Ren, Chao Huang
arXiv AI
Oct 9

LiteGUI: Lightweight GUI Agents via Multi-Solution Guided Distillation and Dual-Level Reinforcement Learning

LiteGUI is a new framework for training lightweight GUI agents that tackles the long‑horizon, multi‑path nature of GUI interactions. It introduces a two‑stage post‑training pipeline: Guided On‑Policy Distillation, which uses human‑verified multi‑solution action annotations to provide privileged teacher guidance at each state, and Multi‑Solution Dual‑Level GRPO, which combines action‑level supervision with history‑conditioned planning‑quality supervision while handling multiple valid actions. The authors also present a scalable data generation pipeline and the Lite‑Dataset, enabling extensive evaluation across GUI benchmarks and demonstrating substantial performance gains over existing training paradigms.

By Yubin Wu, Zicheng Cai, Liping Ning, Hua Wang, Zhi Chen, Yaohua Tang, Hao Chen