Continual Learning Exam: Benchmarking Long-Horizon RSI and Experiential Learning in LLM Agents
Read the original on arXiv AI →The paper introduces the Continual Learning Exam (CLE), a benchmark designed to evaluate how large language model (LLM) agents improve over time by learning from experience. CLE consists of 120 human-authored tasks organized into five-level ladders, totaling 600 levels across 89 professional roles in 12 domains, where agents must pass each level to unlock the next. Experiments with 17 frontier LLM configurations show that providing diagnostic feedback and allowing one revision improves median completion by 5.3 percentage points, yet the best system only clears 47.7% of levels and completes all five levels in 23.3% of tasks, underscoring the challenge of sustained learning from experience.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.