Learn2Play Bench is a new benchmark that tests how well large language model agents learn from experience in unfamiliar, text‑based games with novel or counterintuitive rules. The benchmark provides reproducible feedback, automatic scoring, and varied game instances to evaluate learning across repeated attempts and transfer to new situations. Findings show that retaining full action records aids learning, human players outperform agents, and the choice of harness significantly impacts performance and inference cost.
By Yibo Li, Jinhang Qiu, Zhi Zheng, Qianyun Guo, Jiaying Wu, Shuo Ji, Bryan Hooi
AgentOdyssey is a new evaluation framework that procedurally generates open-ended text games with rich entities, dynamic worlds, and long-horizon tasks to test test-time continual learning agents. The framework measures game progress and provides diagnostic tests on world knowledge acquisition, episodic memory, exploration, action diversity, and model cost. Experiments across six games show that even the best agents fall far short of human performance, and supervised fine-tuning leads to catastrophic forgetting, highlighting the need for better parametric continual learning methods.
By Zheyuan Zhang, Zehao Wen, Alvin Zhang, Andrew Wang, Jianwen Xie, Daniel Khashabi, Tianmin Shu
The paper introduces EDV, an Execute‑Distill‑Verify framework that constructs offline experience for large language model agents without updating model parameters. In EDV, multiple heterogeneous agents execute tasks, a distiller extracts candidate lessons from their trajectories and feedback, and executors review these lessons, keeping only those unanimously approved. Experiments on τ²‑bench, Mind2Web, and MMTB demonstrate improved performance on revisited tasks and within‑domain transfer, while a Retail subset evaluation shows EDV cuts the Self‑Confirmation Trap Rate from 11.0% to 5.0%.
By Shiding Zhu, Yudi Qi, Yajie Wang, Jiaze Li, Chao Song, Yaorui Shi, Yibo Miao, Hanqi Gao, Kai Zhang
Agent0 is a fully autonomous framework that enables large language model agents to evolve without external data by using a multi‑step co‑evolution process. It pits a curriculum agent against an executor agent, both derived from the same base LLM, where the curriculum agent creates increasingly challenging tasks and the executor learns to solve them. By integrating external tools into the executor’s workflow, the system creates a self‑reinforcing cycle that continuously generates high‑quality curricula, leading to significant gains in reasoning performance—an 18% improvement on mathematical reasoning and 24% on general reasoning for the Qwen3‑8B‑Base model.
By Peng Xia, Kaide Zeng, Jiaqi Liu, Can Qin, Fang Wu, Yiyang Zhou, Caiming Xiong, Huaxiu Yao
The paper introduces Randomized Block Curation (RBC), a method for selecting useful lessons from language‑model agents’ persistent textual memory. RBC evaluates the impact of grouped lessons through randomized experiments on practice tasks, retaining only those that demonstrably improve task success. Experiments across two memory systems, three models, and three benchmarks show RBC raises average success to 66.9, outperforming both full playbooks and agent‑chosen lessons.
arXiv:2608. 15071v1 Announce Type: new Abstract: Learning from experience is critical for developing capable, self-improving large language model (LLM) agents.
By Tianxin Wei, Zhan Shi, Minhua Lin, Bing He, Zewen Liu, Yisi Sang, Yuanchen Bei, Xuying Ning, Jiaru Zou, Ting-Wei Li, Xiao Lin, Yanjun Zhao, Chi Wang, Benoit Dumoulin, Dakuo Wang, Jingrui He, Hanqing Lu