arXiv AI

SimSkill: A Lifelong Learning AI Agent for Autonomous Mastery of Traffic Simulation

SimSkill is a lifelong learning AI agent that uses the SUMO traffic simulator to autonomously identify gaps in its capabilities, generate and solve tasks grounded in the environment, and verify solutions through an action‑critic loop. It consolidates experience into episodic, procedural, and semantic memory without updating its backbone language model, creating a reusable library for traffic‑simulation workflows. Evaluations on two benchmarks with three different LLM backbones show that SimSkill can improve verified completion rates by up to 25 percentage points, with procedural and semantic memory contributing complementarily to performance.

arXiv AI

Sep 12

SimSkill: A Self-Evolving LLM Agent for Skill and Knowledge Accumulation in Traffic Simulation

SimSkill is a self‑evolving large‑language‑model agent designed for the SUMO traffic simulator. It continuously detects capability gaps, creates and solves environment‑grounded tasks, verifies solutions via an action–critic loop, and stores experiences in episodic, procedural, and semantic memory. Evaluations on two held‑out benchmarks across three LLM backbones show up to a 25‑percentage‑point improvement in verified success, with procedural and semantic memory contributing complementarily.

By Qi Liu, Qinzheng Wang, Can Li, Yiming Bie, Wanjng Ma

llms agents benchmarks

arXiv AI

Jul 1

Long-term Traffic Simulation via Structured Autoregressive Modeling

arXiv:2606. 31209v1 Announce Type: new Abstract: Interactive traffic simulation is a vital world model for autonomous driving.

By Lingyu Xiao, Zexin Feng, Xintao Yan

llms agents robotics benchmarks safety

arXiv AI

Jun 2

TrafficClaw: A Generalizable LLM Agent in the Unified Physical Environment for Urban Traffic Control

arXiv:2604. 17456v2 Announce Type: replace Abstract: Large language model (LLM) agents have shown strong capabilities in long-horizon reasoning, tool use, and decision-making in digital environments, yet extending them to physically grounded systems remains challenging.

By Siqi Lai, Pan Zhang, Yuping Zhou, Jindong Han, Yansong Ning, Hao Liu

llms agents reinforcement-learning

arXiv AI

Aug 28

SKILL.state: Scalable Long-Horizon Agent Skills

SKILL.state is a new runtime architecture for large language model agents that replaces the traditional append‑only conversational history with an explicit, mutable execution state. At each step the model receives only the immutable skill specification, the current structured state, and the latest observation, discarding intermediate reasoning after validating state updates. Experiments across datasets, models, and environments show that SKILL.state improves task accuracy and significantly reduces cumulative token consumption, proving that explicit execution state is a scalable, architecture‑agnostic abstraction for long‑horizon agent skills.

By Sanket Badhe, Priyanka Tiwari, Jonghyun Chung

llms agents

arXiv AI

Jun 9

Language-based Trial and Error Falls Behind in the Era of Experience

arXiv:2601. 21754v3 Announce Type: replace Abstract: While Large Language Models (LLMs) excel in language-based agentic tasks, their applicability to unseen, nonlinguistic environments (e.

By Haoyu Wang, Guozheng Ma, Shugang Cui, Yilun Kong, Haotian Luo, Li Shen, Mengya Gao, Yichao Wu, Xiaogang Wang, Dacheng Tao

llms agents reinforcement-learning fine-tuning

arXiv AI

Jul 21

Environment-free Synthetic Data Generation for API-Calling Agents

arXiv:2607. 16900v1 Announce Type: new Abstract: Training API-calling large language model (LLM) agents demands massive amounts of high-quality trajectories.

By Seanie Lee, Sanjoy Chowdhury, Chao Jiang, Cheng-Yu Hsieh, Ting-Yao Hu, Alexander T Toshev, Oncel Tuzel, Raviteja Vemulapalli

llms agents fine-tuning benchmarks

arXiv AI

Aug 13

One Frozen Simulator Is Not Enough: Simulator Collapse in Multi-Agent RL

arXiv:2608. 12253v1 Announce Type: cross Abstract: Multi-agent reinforcement learning for human-AI interaction typically relies on a single large language model to simulate user behavior.

By Simon Yu, Nicholas Tomlin, Marwa Abdulhai, Ximing Lu, Derek Chong, Abe Hou, Dilara Soylu, Sergey Levine, Christopher D. Manning, Weiyan Shi

llms agents reinforcement-learning benchmarks

arXiv AI

Aug 18

Evo-Harness: Context-to-Harness Skill Compilation for Self-Evolving Agents

arXiv:2608. 15071v1 Announce Type: new Abstract: Learning from experience is critical for developing capable, self-improving large language model (LLM) agents.

By Tianxin Wei, Zhan Shi, Minhua Lin, Bing He, Zewen Liu, Yisi Sang, Yuanchen Bei, Xuying Ning, Jiaru Zou, Ting-Wei Li, Xiao Lin, Yanjun Zhao, Chi Wang, Benoit Dumoulin, Dakuo Wang, Jingrui He, Hanqing Lu

llms agents benchmarks

arXiv AI

Jul 15

TerraZero: Procedural Driving Simulation for Zero-Demonstration Self-Play at Scale

arXiv:2607. 13028v1 Announce Type: cross Abstract: Training robust autonomous driving agents requires a simulator that is fast enough for reinforcement learning at scale, realistic enough to ground behavior in real-world map structure, and diverse enough to cover the safety-critical long tail that logged data rarely contains.

By Zhouchonghao Wu, Akshay Rangesh, Weixin Li, Wei-Jer Chang, Zachary Lee, Tim Wang, Wei Zhan

agents reinforcement-learning robotics benchmarks safety

arXiv AI

Jul 15

Mobility-Aware Cache Framework for Scalable LLM-Based Human Mobility Simulation

arXiv:2602. 16727v2 Announce Type: replace Abstract: Simulating large-scale human mobility is fundamental to understanding population movement patterns and supporting real-world geospatial applications such as urban planning, epidemic response, and transportation analysis.

By Hua Yan, Heng Tan, Yingxue Zhang, Yu Yang

llms rag agents efficiency benchmarks

arXiv AI

Jul 28

CoopReflect: Towards Natural Language Communication for Cooperative Autonomous Driving via Multi-Agent Learning

arXiv:2505. 18334v2 Announce Type: replace-cross Abstract: Past work has demonstrated that autonomous vehicles can drive more safely if they communicate with each other.

By Jiaxun Cui, Chen Tang, Jarrett Holtz, Janice Nguyen, Alessandro G. Allievi, Hang Qiu, Peter Stone

llms agents robotics

arXiv AI

Jul 13

Improving Language Agents through BREW: Bootstrapping expeRientially-learned Environmental knoWledge

arXiv:2511. 20297v2 Announce Type: replace Abstract: Large Language Model (LLM)-based agents are increasingly capable of complex, multi-step tasks such as GUI automation, tool use, and data manipulation, yet they cannot learn from experience: each new session rediscovers solutions from scratch.

By Shashank Kirtania, Param Biyani, Priyanshu Gupta, Yasharth Bajpai, Roshni Iyer, Sumit Gulwani, Gustavo Soares

llms rag agents robotics benchmarks

Related stories