arXiv:2605.27995v3 Announce Type: replace
Abstract: Large language model (LLM)-based agents have shown strong capabilities in using external tools to solve complex tasks. However, existing evaluation...
By Kou Shi, Ziao Zhang, Shiting Huang, Avery Nie, Zhen Fang, Qiuchen Wang, Lin Chen, Huaian Chen, Zehui Chen, Feng Zhao
arXiv:2608. 02650v1 Announce Type: new Abstract: Large language model (LLM) agents increasingly rely on external tools to complete complex real-world tasks.
By Zian Zhai, Xingyu Tan, Gaowang Zou, Xiaoyang Wang, Wenjie Zhang
arXiv:2607. 05174v1 Announce Type: new Abstract: Language agents, i.
By Zhiheng Xi, Dingwen Yang, Jiaqi Liu, Jixuan Huang, Honglin Guo, Baodai Huang, Tinggang Chen, Qi Zhang, Zhonghang Lu, Chenyu Liu, Jiajun Sun, Jiazheng Zhang, Dingwei Zhu, Xin Guo, Junzhe Wang, Zhihao Zhang, Yuming Yang, Junjie Ye, Minghe Gao, Dongrui Liu, Jiaming Ji, Guohao Li, Tao Gui, Qi Zhang, Xuanjing Huang
The paper introduces TURTLE, an Unsupervised Environment Design framework that evolves tool‑use tasks for AI assistants by using a proposer, solver, and feedback generator powered by large language models. TURTLE automatically generates self‑tuning, long‑horizon benchmarks that increase task difficulty, diversity, and failure modes across multiple applications. Experiments on AppWorld, τ‑Bench, and AutomationBench demonstrate significant gains in hard task percentages and broader reasoning challenges.
By Hyunji Alex Nam, Michael Dennis, Natasha Jaques, Minsuk Chang
CoworkOS is a general‑purpose harness designed for long‑horizon real‑world automation, separating task strategy from execution management. Its three‑layer architecture allows direct execution or delegation to subagents, while a shared application layer lets humans and agents collaboratively edit documents, designs, and 3D scenes. The system incorporates hierarchical workspaces and persistent semantic memory to maintain continuity across agents and sessions, and demonstrates strong performance across 12 benchmarks and six application categories, outperforming leading agents like Codex and DeepSeek Harness.
By Jiajie Jin, Guanting Dong, Chengjun Pan, Xin Deng, Zhuohui Sheng, Guoqiang Wang, Tao Gui, Yutao Zhu, Yujia Chen, Shihong Yi, Binze Hu, Zhixiong Yang, Zhichao Sun, Rui Wang, Han Han, Mingyu Ma, Junhao Huang, Xiaotao Wang, Huicheng Jiang, Yuyang Hu, Xinyu Yang, Wanghan Xu, Yajie Yang, Zhecan Wang, Honglin Guo, Xiaoyu Xing, Xiaoqian Zhang, Yulun Zhu, Xiaoxi Li, Jiahang Lin, Shaofan Liu, Shihan Dou, Zhiheng Xi, Qi Zhang, Yu-Gang Jiang, See-Kiong Ng, Yang Zhang, Tat-Seng Chua, Ji-Rong Wen, Zhicheng Dou
arXiv:2609.06059v1 Announce Type: new
Abstract: As large language models evolve from question-answering systems into general-purpose agents, evaluation must move beyond static answer correctness to a...
By Yu Liu, Zhilin Liu, Zhiwei Yang, Shaojie Zhang, Zheyuan Deng, Tingwei Huang, Zhenbo Luo, Lei Jiang, Yanbing Liu, Pei Fu