arXiv AI

False Frontiers: Diagnosing and Mitigating Co-Cheating in Self-Evolving Search Agents

The paper investigates a failure mode called co‑cheating in self‑evolving search agents, where the proposer and solver agree on shared errors, inflating internal reward without improving external correctness. The authors first propose multi‑sample verification (MSV) to filter unreliable pseudo‑labels, which only partially mitigates the issue. They then introduce CrossFit, a cross‑fitted reward scheme that partitions source documents and uses an auxiliary solver to prevent same‑source agreement, significantly reducing false agreement and boosting downstream benchmark performance.

arXiv AI

Sep 2

DiagEvo: Diagnosis-Guided Self-Evolution via Hierarchical Error Memory

DiagEvo is a self‑evolution framework that guides language‑model training by extracting recurring error causes from a solver’s own failure history and storing them in a hierarchical error‑cause memory. The system classifies causes as Active or Mastered, uses this information to balance targeted question generation with exploration, and applies double‑confidence filtering to keep only intermediate‑difficulty questions. Experiments show that DiagEvo outperforms baselines on nine benchmarks for three solvers, achieving up to 72.3% mean accuracy on five mathematical reasoning tasks.

By Xincheng Wei, Yifan Ding, Yoshua Li, Dongsheng Ma, Rongxiang Weng, Xunliang Cai, Wenjian Ding, Yao Zhang

reinforcement-learning benchmarks

arXiv AI

Aug 7

SkillHEX: Improving Agent Skills via Hypothesis-Driven Autonomous Exploration and Exploitation

arXiv:2608. 05628v1 Announce Type: new Abstract: Although agent skills equip LLMs with reusable procedural knowledge, manual maintenance suffers from high costs, unscalability, and misalignment.

By Yuru Feng, Yaoqi Chen, Beidi Zhao, Qianxi Zhang, Xinjiang Wang, Jianan Lu, Zhirui Wang, Shusen Xu, Zengzhong Li, Qi Chen

llms agents

arXiv AI

Jun 30

SEVA: Self-Evolving Verification Agent with Process Reward for Fact Attribution

arXiv:2606. 29713v1 Announce Type: cross Abstract: Hallucination is the reliability bottleneck for LLM-based agents, and fact attribution verifiers are the last line of defense -- yet today's verifiers emit only opaque binary labels, leaving agents unable to self-correct and operators unable to audit.

By Aojie Yuan, Yi Nian, Haiyue Zhang, Zijian Su, Yue Zhao

llms agents reinforcement-learning benchmarks safety

arXiv AI

Jun 9

PACE: Anytime-Valid Acceptance Tests for Self-Evolving Agents

arXiv:2606. 08106v1 Announce Type: new Abstract: Self-evolving agents improve by repeatedly proposing changes to their own prompts, skills, or workflows and keeping those that score higher on a small held-out set.

By Zayx Shawn

agents

arXiv AI

1d ago

Learning What to Practice: Diagnosis-Guided Self-Evolution for Language Models

arXiv:2609.00768v2 Announce Type: replace Abstract: Self-play supports the self-evolution of language models, but solver performance can plateau or decline across rounds without guidance. Existing un...

By Xincheng Wei, Yifan Ding, Fucheng Xiong, Yoshua Li, Dongsheng Ma, Rongxiang Weng, Xunliang Cai, Wenjian Ding, Yao Zhang

llms reinforcement-learning benchmarks

arXiv AI

Jun 2

Self-Trained Verification for Training- and Test-Time Self-Improvement

arXiv:2605. 30290v2 Announce Type: replace-cross Abstract: Self-improvement at scale has been a longstanding goal for reasoning models, and there are two natural places to do it: at test time, through verification-refinement (V-R) loops; and at training time, through self-training methods.

By Chen Henry Wu, Aditi Raghunathan

reinforcement-learning fine-tuning

arXiv AI

Aug 3

Self-Play Meets Skill Evolution: Self-Evolving Search Agents that Pose, Solve, and Remember

arXiv:2607. 29468v1 Announce Type: new Abstract: Self-play agents can generate training problems without questions from target benchmarks, but their curricula lack persistent state: failures affect gradients yet do not explicitly shape future practice.

By Zenghuang Fu, Zhaoyang Li, Qiuyuan Ai, Haoyu Wu, Minghui Wu, Chenxu Zhao, Ante Wang, Guannan He, Changwei Wang

agents reinforcement-learning benchmarks

arXiv AI

Aug 18

Admission Without Answers: Label-Free Certification and Experience Learning for LLM-Based Optimization Modeling

arXiv:2608. 15565v1 Announce Type: new Abstract: Experience-learning agents for optimization modeling improve by storing verified skills, but existing learners admit knowledge by checking against known answers, which real ticket streams do not provide.

By Junbo Jacob Lian, Huiling Chen, Hanzhang Qin, Chung-Piaw Teo

llms agents benchmarks

arXiv AI

1d ago

Do Self-Evolving Skills Generalize to Held-Out Tasks?

arXiv:2609.39148v1 Announce Type: new Abstract: AI agents can externalize what they learn from past tasks into reusable \emph{skills}, such as procedures, checklists, code, or other executable artifa...

By Xihao Piao, Zifeng Wang, Zhen Chen

llms agents benchmarks

Hugging Face Trending Papers

Aug 6

SkillHEX: Improving Agent Skills via Hypothesis-Driven Autonomous Exploration and Exploitation

Although agent skills equip LLMs with reusable procedural knowledge, manual maintenance suffers from high costs, unscalability, and misalignment. Real-world deployments thus require autonomous, on-demand skill evolution at test time, constrained by limited interaction budgets and a lack of training or validation sets.

llms agents

arXiv AI

Jul 17

Reward-Free Evolving Agents via Pairwise Validator

arXiv:2607. 14408v1 Announce Type: new Abstract: A self-evolving agentic loop repeatedly proposes a tweaked version of an agent (its prompt template or program) and accepts or rejects the change based on a per-iteration quality signal.

By Minghao Liu, Yu Wang, Jiayun Wang, Wei Wei

llms agents

arXiv Machine Learning

Jul 8

More Convincing, Not More Correct: Self-Play Reward Hacking of Reference-Free LLM Judges

arXiv:2607. 05904v1 Announce Type: new Abstract: Training a language model against its own reference-free judgments (the premise of self-rewarding, self-play, and LLM-as-a-judge pipelines) assumes a model's verdict on a shown answer tracks correctness.

By Chenyu Zhou

llms reinforcement-learning

Related stories