arXiv:2607. 12790v1 Announce Type: new Abstract: Self-evolving agent systems improve by creating, revising, and retiring their own skills, but every such loop rests on a hidden assumption: a reliable evaluation metric already exists.
By Xing Zhang, Guanghui Wang, Yanwei Cui, Ziyuan Li, Wei Qiu, Bing Zhu, Peiyang He
Agentic systems have widened the gap between producing candidate outputs and reviewing them. This paper asks a practical architectural question: should domain specialization be built into an evaluator's weights, or into the rule that decides when its judgment can be trusted?
The paper argues that using a large language model (LLM) as the sole judge in self‑improving agent pipelines is problematic, as the judge can be biased or manipulated, leading to false confidence in system performance. The authors propose a new framework, PROCTOR, which replaces the oracle judge with a deterministic, teacher‑student loop that enforces guardrails such as sandboxing, role separation, and acceptance checks to prevent cheating and ensure reliable evaluation. Experiments across contract analysis, compliance review, and code quality demonstrate that PROCTOR mitigates eleven identified failure modes that previously allowed agents to achieve perfect scores while hiding significant capability gaps.
By Vansh Wahi
A self-evolving agent retires its bad skills by watching them fail, so what happens when the judge cannot see the failures? Skill retirement is the structural constraint that keeps a growing library from drifting below the no-skill baseline, but its guarantee assumes an unbiased reward, which is false for the LLM judges that reference-free tasks force upon us.
arXiv:2605. 22148v3 Announce Type: replace Abstract: A large language model (LLM) agent that writes and edits its own skill library must also decide which skills to keep, from one noisy scalar per skill.
By Xing Zhang, Yanwei Cui, Guanghui Wang, Ziyuan Li, Wei Qiu, Bing Zhu, Peiyang He
arXiv:2607. 07436v1 Announce Type: new Abstract: A self-evolving agent retires its bad skills by watching them fail, so what happens when the judge cannot see the failures?
By Xing Zhang, Yanwei Cui, Guanghui Wang, Ziyuan Li, Wei Qiu, Bing Zhu, Peiyang He