CAFE (Coupled Agent–Feedback Evolution) is a framework that lets a shared‑parameter model alternate between acting as a search agent and as a critic that provides corrective feedback. By learning when to request feedback and how to use it, CAFE trains the agent to recover from its own failures and shapes rewards both online and offline. Experiments on seven search benchmarks show that CAFE outperforms other RL‑based agents, maintains gains on out‑of‑domain tests, and reduces hallucinations, indicating that co‑evolving feedback is essential for self‑improving search agents.
By Boyang Liu, Senjie Jin, Peixin Wang, Zhangyue Yin, Yibo Wang, Yuhao Zhou, Xinbing Liang, Shizheng Zhu, Yuhui Wang, Jingqi Tong, Zhiheng Xi, Jiazheng Zhang, Clive Bai, Clarenceai, Blaze Chen, Tao Gui, Qi Zhang, Xuanjing Huang
arXiv:2607. 23263v1 Announce Type: new Abstract: Deciding whether a trajectory actually fulfills its instruction governs how we measure computer-use agents on long-horizon graphical-user-interface tasks and how we train them with reinforcement learning.
By Yang Wan, Zhenhao Zhang, Jierui Wang, Linchao Zhu
arXiv:2608. 07531v1 Announce Type: cross Abstract: Search-augmented language agents should retrieve external information only when necessary and ground their answers in retrieved evidence.
By Cheng Ruoxi, Ma Haoxuan, Zhang Hongyi, Zhang Junming, Duan Ranjie, Xia Qiaolin, Wang Hao, Lu Yu, Shi Haibo, Ma Xingjun
As Large Language Model (LLM) agents are applied in continuously interactive environments, driving the evolution of their own capabilities becomes a core problem for achieving long-term autonomy. Curr...
arXiv:2608. 06714v1 Announce Type: new Abstract: Recent systems for optimizing prompts, programs, and ML workflows typically rely on explicit outer-loop controllers such as evolutionary search, bandits, or textual-gradient methods.
By Junbo Li, Boyi Liu, Canwen Xu, Yite Wang, Yuxiong He, Zhangyang Wang, Qiang Liu, Zhewei Yao
arXiv:2609.24289v1 Announce Type: new
Abstract: As Large Language Model (LLM) agents are applied in continuously interactive environments, driving the evolution of their own capabilities becomes a co...
By Ruimin Pei, Yongkang Wu, Shangyi Zheng, Yaqing Zhang, Deyang Li, Jianjun Tao, Xinyu Zhang, Xiang Zhang
Recent systems for optimizing prompts, programs, and ML workflows typically rely on explicit outer-loop controllers such as evolutionary search, bandits, or textual-gradient methods. We ask a fundamentally different question: how much of this search policy can be internalized by a single tool-using agent?
Search-G1 is a reward framework that encourages language agents to retrieve external information only when necessary and to ground their answers in that evidence. It uses two hidden‑state readouts—one predicting whether a closed‑book answer suffices and another measuring sensitivity to evidence deletion—to calibrate retrieval necessity and evidence reliance. The reward system gates evidence credit by necessity, rewards sufficient direct answers, and penalizes redundant searches, all without requiring costly annotations or LLM‑judge inference.
By Ruoxi Cheng, Haoxuan Ma, Hongyi Zhang, Junming Zhang, Ranjie Duan, Qiaolin Xia, Hao Wang, Yu Lu, Haibo Shi, Xingjun Ma
RSI-Opt introduces Recursive Self-Improvement Optimizer Agents that evolve their own training policies by accumulating knowledge from previous runs, including prompts, playbooks, optimizer code, and decision cadences. A meta‑agent rewrites and archives these policies, enabling reusable procedures and code that adapt to new tasks. Across multiple generations and tasks, RSI‑Opt improves optimizer performance, achieving higher z‑scores on evolution suites, better cross‑model transfer, and significant error reductions on PDE benchmarks compared to fixed‑prompt baselines.
By Binghang Lu, Yige Sun, Guang Lin
arXiv:2605. 29796v3 Announce Type: replace Abstract: Agentic search enables LLMs to solve complex multi-hop questions through iterative reasoning and external search.
By Yunbo Tang, Chengyi Yang, Shiyu Liu, Zhishang Xiang, Zerui Chen, Qinggang Zhang, Jinsong Su
The paper introduces Online Meta-Control of Experience (OMCE), a framework that updates an agent’s experience memory and learns a meta-control policy during test-time adaptation. OMCE uses a lightweight contextual-bandit meta-controller to choose among four cognitive actions—DIRECT, EPISODE, GUIDE, and DISTILL—based on compact retrieval signals, thereby deciding when to invoke experience, which memory format to use, and when to consolidate new guidance. Experiments on four online adaptation benchmarks across four backbones show that OMCE can improve first-attempt success by up to 20.5 percentage points over the strongest baseline while reducing per-task token use by a factor of 5.7 to 62.5 relative to ACE with GPT‑5.2, and controlled ablations confirm the benefits of learned routing and ongoing policy adaptation.
By Zeyuan Zhao, Yuanyuan Shi, Peng Yu, Lingyue Fu, Shao Zhang, Ying Wen
The paper investigates reflective prompt optimization (GEPA) for a multi‑agent deep‑research language‑model, focusing on a topic‑disjoint train/validation/test split to avoid memorization. Across three random seeds, the optimized prompts improve held‑out citation reward from 0.702 to 0.781, with a 95% confidence interval of [0.051, 0.107]. The study shows that the gain is driven by decomposed per‑dimension feedback rather than a single scalar reward, and that the improvement persists even with an out‑of‑family verifier and a hand‑written version of the discovered recipe.
By Elias Lumer, Sahil Sen, Vamse Kumar Subbiah, Kevin Paul