arXiv AI
Auditing the Product, Not the Recursion: The Evaluation Evidence Behind Recursive Self-Improvement Claims
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
Hugging Face Trending Papers
Oct 20
Auditing the Product, Not the Recursion: The Evaluation Evidence Behind Recursive Self-Improvement Claims
Recursive self-improvement (RSI) promises AI systems, such as self-evolving LLM agents, that become better at producing further improvements. Evaluating this promise requires separating product claims...
arXiv AI
Jul 31
One Run Is Not an Idea: The Implementation Lottery in Automated Research
arXiv:2607. 26587v1 Announce Type: cross Abstract: Automated research systems use experimental scores both to deliver artifacts and to decide which ideas to retain, transfer, and pursue.
arXiv AI
Oct 2
From Proposal to Verified Effect: Praxa, an Evidence-Bound Harness for Governed AI Agent Execution
Praxa is an evidence‑bound harness for governed AI agent execution that explicitly represents states such as proposal, authority, dispatch, verified external effect, and promotion through deterministic admission, brokered execution, external read‑back, reconciliation, and reviewed promotion. The authors report four evidence lanes: a repository‑local audit passing all unit and Workerd tests; a pilot on 12 curated tasks where both baseline and reliability‑layer arms passed 17 of 36 trials; a coordination‑proxy comparison where both baseline and a source‑authored candidate completed all 180 trials with equal accuracy but the candidate used fewer tokens and steps; and deployed source/configuration evidence showing bounded reflection, recall accounting, memory compilation, and tool‑health paths. None of the evidence demonstrates superiority in security, safety, or user benefit. whyItMatters:"Praxa provides a testable architecture that makes authority‑to‑effect transitions explicit, offering a framework for verifying AI agent behavior, though current evidence does not prove improved security or performance."
arXiv AI
1d ago
Beneath the Surface: What We Overlook When We Talk About Agent Evolution
The paper investigates how agent harness evolution supports recursive self‑improvement without changing model weights, assessing four methods across general and coding tasks. Repeated tests and component interventions show that all evolved agents acquire new tasks while losing some previously mastered ones, with reliability gains sometimes offset by higher deployment costs. The study highlights that the highest‑scoring agent does not always deliver the greatest mean improvement, and that stricter admission criteria can eliminate useful alternatives; it proposes preservation‑aware evaluation and a follow‑up tool, EvoAudit, to refine agent design.
arXiv AI
Oct 19
EvalResearchBench: Can AI Agents Build Their Own Feedback for Recursive Self-Improvement?
EvalResearchBench (ERB) is a benchmark that tests whether AI agents can autonomously design evaluation tasks for recursive self‑improvement. In ERB, a researcher agent selects or synthesizes tasks, builds graders, and refines them through pilot tests before producing a final evaluator that runs within fixed time and API budgets. Experiments with nine researchers and thirteen candidate models show that the best automated evaluators achieve about 75% concordance with target benchmarks, below the 91% ceiling set by target disagreements, and that human‑designed task samples still outperform many automated approaches.
arXiv AI
Oct 27
Trustworthy Recursive Self-Improvement in the Era of Large Models: Foundation, Risks, Evaluation and Open Challenges
arXiv:2610.35193v1 Announce Type: new Abstract: AI systems built around large models increasingly use experience and feedback to make persistent changes to models, prompts, memories, tools, and workf...