The paper introduces the Continual Learning Exam (CLE), a benchmark designed to evaluate how large language model (LLM) agents improve over time by learning from experience. CLE consists of 120 human-authored tasks organized into five-level ladders, totaling 600 levels across 89 professional roles in 12 domains, where agents must pass each level to unlock the next. Experiments with 17 frontier LLM configurations show that providing diagnostic feedback and allowing one revision improves median completion by 5.3 percentage points, yet the best system only clears 47.7% of levels and completes all five levels in 23.3% of tasks, underscoring the challenge of sustained learning from experience.
arXiv:2609.05435v2 Announce Type: replace Abstract: Can language agents continually learn from experience, turning earlier interactions into reusable capabilities? AhaBench evaluates this ability thr...
arXiv:2605.08693v3 Announce Type: replace Abstract: Skills provide an effective mechanism for improving LLM agents on complex tasks, yet in existing agent frameworks, their creation, refinement, and...
Modern agent frameworks equip large language models with external skill libraries to solve complex tasks. However, it remains unclear whether these systems can effectively evolve their skills and whether the resulting skills improve task-solving capabilities.
arXiv:2608. 03874v1 Announce Type: new Abstract: Modern agent frameworks equip large language models with external skill libraries to solve complex tasks.
arXiv:2606. 01139v1 Announce Type: new Abstract: Agent skills are procedural artifacts that enable LLM agents to execute workflows, verify constraints, and recover from failures.
arXiv:2609.23989v1 Announce Type: new Abstract: Building general-purpose agents for industrial deployment requires integrating multiple capabilities, each typically acquired at a distinct stage of tr...
arXiv:2610.34043v1 Announce Type: cross Abstract: Autonomous agents are increasingly deployed across sequences of related tasks, yet they often fail to retain and improve their knowledge over time. E...
Autonomous agents are increasingly deployed across sequences of related tasks, yet they often fail to retain and improve their knowledge over time. Existing approaches convert interaction trajectories...
SkillGym is an automatic pipeline that generates verifiable environments for training skill-use agents. It crawls internet skills, filters for reproducible workflows, and uses a builder‑reviewer process to create difficulty‑controlled tasks with reference solutions and verifiers. The system builds 6.8k environments, collects 19k successful trajectories, and fine‑tunes LLMs from 2B to 122B parameters, improving performance and skill invocation rates.
The paper introduces RIKA, a method for reliable knowledge acquisition in continual learning agents. RIKA hierarchically extracts factual, conceptual, procedural, and metacognitive knowledge and uses evidence-based admission to decide which claims are incorporated into the agent’s context, avoiding the reuse of unreliable information. Experiments on six task streams and four model backbones show that RIKA outperforms existing memory and experience-learning systems on aggregate performance.
arXiv:2610.07792v1 Announce Type: cross Abstract: Large language model agents are increasingly deployed to perform complex tasks in real-world environments. However, the knowledge required for correc...
AgentOdyssey is a new evaluation framework that procedurally generates open-ended text games with rich entities, dynamic worlds, and long-horizon tasks to test test-time continual learning agents. The framework measures game progress and provides diagnostic tests on world knowledge acquisition, episodic memory, exploration, action diversity, and model cost. Experiments across six games show that even the best agents fall far short of human performance, and supervised fine-tuning leads to catastrophic forgetting, highlighting the need for better parametric continual learning methods.