arXiv AI By Yucheng Jiang, Zora Zhiruo Wang, Ruishi Chen, Diyi Yang

Inducing Task Models from Computer-Use Traces

Read the original on arXiv AI →

arXiv:2608. 20319v1 Announce Type: cross Abstract: Naturalistic computer-use traces, passively recorded screenshots and mouse or keyboard actions, are a valuable resource for deriving symbolic, auditable, and reusable models of how everyday work is done.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Oct 12

Work2Skill: Capturing Human GUI Experience for Reusable Agent Skill Induction

Work2Skill is a framework that captures human GUI experience to induce reusable agent skills. It introduces WorkRecorder, an infrastructure that aligns screen recordings, mouse and keyboard events, and accessibility-tree data, and uses it to build WorkDemo-350, the first synchronized multimodal dataset of 350 human desktop GUI demonstrations. The proposed skill‑induction method extracts workflow rules, step dependencies, and completion checks, achieving higher scores and success rates than baseline methods on MyPCBench and OSWorld benchmarks.

By Yuqi Huang, Hui Li, Yifei Cao, Zikun Cai, Tianci Xie, Yangfan Gao, Pinxue Guo, Chaoyun Zhang, Bingxuan Li, Rui Zheng, Hang Yan, Ning Liao
arXiv AI
Oct 15

SkillHorizon: Constructing Long-Horizon Agent Tasks through Skill Composition

arXiv:2610.14966v1 Announce Type: new Abstract: Long-horizon agents must complete workflows in which later activities depend on earlier outputs and changes to shared state. A key challenge for evalua...

By Yuxuan Liu, Zhaochen Su, Hangyu Guo, Shijue Huang, Zhixiong Zhang, Kawai Chung, Zhongwei Xie, Tianshi Zheng, Wei Fan, Qing Zong, Haochen Shi, Baixuan Xu, Haoran Li, Yangqiu Song