The paper introduces the Procedural Graph, a framework that structures procedural knowledge into (procedure, relation, procedure) triplets to guide large language model agents in planning and tool usage. At each decision point, a guidance model uses the local subgraph to bias the agent’s next action, while an LLM refiner self‑evolves the graph by editing its topology based on successful versus failed trajectories. Experiments across datasets and LLMs show that Procedural Graphs consistently outperform memory‑based baselines, and the self‑evolution mechanism further improves performance without manual engineering.
arXiv:2607. 13884v1 Announce Type: new Abstract: Large Language Model (LLM) agents have shown remarkable capabilities in autonomous decision-making by generating sequential trajectories of states, actions, and observations.
By Wenjun Wang, Yuchen Fang, Fengrui Liu, Zibo Liang, Kai Zheng
arXiv:2608. 10504v1 Announce Type: new Abstract: As coding agents increasingly handle implementation, the central challenge shifts from building individual agents to building an infrastructure that systematically improves them.
By Jung Hwan Lee, Kyu Ho Lee, Gwang Hoon Yoo
MultiEvoSkill is a skill‑optimization framework that transforms skill documents into typed graphs, enabling explicit procedural topology and parallel branches. It employs a heterogeneous agent system to generate revision suggestions based on diverse experiences, and a structure‑aware debate mechanism to resolve edits while respecting dependencies and importance constraints. Experiments show that this approach outperforms state‑of‑the‑art methods and improves cross‑model generalization of skill artifacts.
By Kounianhua Du, Jianxing Liu, Ruoshui Shan, Yucheng Zhang, Ke Li, Xing Sun, Weinan Zhang
The paper investigates how large language model (LLM) agents can improve performance by storing and reusing experience in external memory. It identifies a problem called error reproduction, where mistakes in stored abstractions propagate through cycles of abstraction, execution, and re-abstraction, leading to persistent errors. To mitigate this, the authors propose Anti-Persistence, a memory design that retains raw trajectories and outcomes but reconstructs task-specific guidance on demand, discarding it after use. Experiments across multiple benchmarks show that Anti-Persistence yields higher held‑out success rates and lower LLM‑call costs compared to other memory designs, supporting the idea of separating experience accumulation from abstraction persistence.
By Ruhan Wang, Kishan Panaganti, Dongruo Zhou
Rep2Skill introduces a representation-guided framework that enables large language model agents to self-evolve their textual skills by analyzing internal representation trajectories from agent rollouts. The method identifies execution turns that deviate from successful dynamics and uses these signals, together with execution contexts, as actionable feedback for targeted skill revision. Experiments with two open-source LLMs across two agent environments demonstrate that Rep2Skill consistently outperforms purely text-based approaches, showing that incorporating internal representations can enhance agent self-improvement.
By Kaixing Zhang, Changming Li, Yingdong Shi, Zheng Zhang, Kaitao Song, Wenjie Shi, Jingang Wang, Kan Ren
The paper introduces SkillPivot, a framework that guides large language model agents to evolve their skills by pinpointing the exact moment a useful problem‑solving sequence turns into an erroneous suffix. SkillPivot uses execution validity, goal progress, and action diversity to detect this deviation point, then employs a stronger teacher to generate a successful alternative from the same prefix. By contrasting the failed and successful suffixes, the method produces localized, compact skill updates that preserve existing effective guidance, outperforming other skill‑evolution techniques on benchmarks such as ToolQA, LogicBench, and WildClawBench.
By Yichun Feng, Jiawei Wang, Haozhe Sun
Memento 3 is a frozen large‑language‑model agent that learns explicit world models by maintaining a natural‑language rulebook in external memory. The agent iteratively observes, reflects, revises the rulebook, compiles it into executable code, and verifies updates through prediction error and replay, accepting only code that the LLM deems faithful to the rulebook. In experiments, a single‑model version solved all 25 public ARC‑AGI‑3 games with perfect efficiency, and a learned feedback controller achieved perfect scores in Atari Pong without further LLM calls.
By Haoyu Zhao, Zhengxu Yu, Zhiyuan He, Meng Fang, Rasul Tutunov, Haitham Bou-Ammar, Weilin Luo, Jun Wang
arXiv:2608. 15071v1 Announce Type: new Abstract: Learning from experience is critical for developing capable, self-improving large language model (LLM) agents.
By Tianxin Wei, Zhan Shi, Minhua Lin, Bing He, Zewen Liu, Yisi Sang, Yuanchen Bei, Xuying Ning, Jiaru Zou, Ting-Wei Li, Xiao Lin, Yanjun Zhao, Chi Wang, Benoit Dumoulin, Dakuo Wang, Jingrui He, Hanqing Lu
arXiv:2607. 01480v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR), along with recent selfdistillation variants such as SDPO, evaluates each rollout against a verifier and updates the policy from that episode-level signal.
By Ye Liu, Srijan Bansal, Bo Pang, Yang Li, Zeyu Leo Liu, Yifei Ming, Zixuan Ke, Shafiq Joty, Semih Yavuz
GraphOPD introduces a graph‑augmented on‑policy distillation method for large language model agents, addressing the shortcomings of existing divergence‑based guidance when rewards are sparse. By constructing a dependency graph from the environment’s state‑change record and applying a random‑walk stationary distribution, the method identifies and prioritizes the most impactful steps in a trajectory, fusing this structural credit with teacher‑student divergence signals. Experiments on ALFWorld, WebShop, and SearchQA demonstrate that GraphOPD consistently outperforms eleven baselines, improving the strongest baseline by up to +5.8 percentage points and showing strong causal alignment through an executed‑replay audit.
By Bohan Lin, Liyi Chen, Zhuoning Guo, Muyang Li, Qimeng Wang, Yan Gao, Yao Hu, Yudong Zhang
PIVOT (Plan-Inspect-eVOlve Trajectories) is a self‑supervised framework that refines large‑language‑model agent plans by treating trajectories as optimizable objects. It operates in four stages—PLAN, INSPECT, EVOLVE, and VERIFY—using structured losses and textual gradients to iteratively improve trajectories and ensure compliance with task constraints. Experiments on DeepPlanning and GAIA show that PIVOT achieves up to 94% relative improvement in constraint satisfaction with human‑in‑the‑loop feedback and still delivers significant gains autonomously, while using 3x to 5x fewer tokens than competing methods.
By Tuo Zhang, Alin-Ionut Popa, Yan Xu, Rui Song, Dimitrios Dimitriadis