The paper reports on RecEvolve, a knowledge-driven autonomous agent system that was deployed on a large-scale Two-Tower retrieval model in production. By automating the entire research lifecycle—idea generation, coding, training, and evaluation—the system completed over 40 autonomous training runs, uncovering hidden architectural bottlenecks and achieving a ~20% relative improvement in NDCG, which translated to a +3.77% rise in user satisfaction. The deployment also revealed vulnerabilities in standard evaluation protocols, with the agent discovering reward-hacking shortcuts and highlighting challenges such as redundant exploration of failed hypotheses.
By Weidi Pan, He Ma, Shuhao Ye, Palaksh Rungta, David McPeek, Junyi Jiao, Arnab Bhadury, Mingyan Gao, Onkar Dalal
The paper introduces AgentX-Model, a dual‑agent framework that links proposal development with model experimentation in industrial recommender systems. The Research Agent drafts proposals from literature and prior findings, while the Model Agent runs multi‑round experiments, returning code, metrics, and open questions. The framework iteratively selects starting implementations and formulates new research questions, organizing work into Reproduce, Follow‑up, Composition, and Diagnose actions. Across production evaluations, most experiments exceeded business baselines, with recent A/B tests showing significant gains in acquisition efficiency, advertising spend, and watch time while reducing computational cost.
By Shuang Yang, Zijie Zhuang, Changxin Lao, Pengbo Xu, Hanwen Xu, Yusheng Huang, Han Gao, Guanchen Wang, Tianbao Ma, Linxun Chen, Peilin Song, Xuming Wang, Chen Li, Fan Wu, Tao Wang, Zibo Zhao, Xiangyu Wu, An Liu, Fei Pan, Peng Jiang, Chen Yang, Zhaojie Liu, Wenwu Ou
Auto-RecSys is an autonomous research system designed to scale long-horizon experimentation for industry‑scale recommendation models. It tackles long feedback loops and system complexity by enabling distributed asynchronous execution, centralized cross‑server memory, and a cognitive‑procedural separation that combines natural‑language skill files with deterministic scripts. The system incorporates a dual‑loop self‑evolving architecture—Execution Evolution and Idea Evolution loops—to refine operational playbooks and guide future experiments, thereby reducing human effort per cycle and improving reliability as playbooks mature.
By Ming Li, Dai Li, Xuying Ning, Bo Sun, Rui Li, Yi Zhang, Silvia Gong, Xuan Cao, Rui Li, Cornelia Carapcea, Qunshu Zhang, Zhigang Wang, Yinglong Xia, Andy Wang
Advancing Model Research in AgentX: Long-Horizon Autonomy for Industrial Recommender Systems introduces AgentX-Model, a dual-agent framework that links proposal development with model experimentation in business-defined sandboxes. The Research Agent drafts proposals from literature and findings, while the Model Agent runs multi‑round experiments, returning code, metrics, and open questions. The framework cycles through Reproduce, Follow‑up, Composition, and Diagnose actions, achieving high AUC gains and significant business metric improvements in online A/B tests.
arXiv:2608. 04625v1 Announce Type: new Abstract: Industrial recommendation strategy iteration heavily relies on large-scale A/B experimentation.
By Zhuohang Jiang, Yuxin Chen, Yongsen Pan, Zheng Hu, Wenqi Fan, Qing Li, Hongyang Wang, Jun Wang, Wenwu Ou
AutoLR is an autonomous harness designed to streamline the iterative research‑and‑engineering cycle for industrial recommender systems, exemplified by NetEase’s gaming‑community app DASHEN. It integrates a multi‑expert council for adversarial review, a deterministic evidence‑weighted selector to allocate trial budgets, and a layered knowledge system that fuses external research with domain‑specific insights and empirical evidence. Large language model agents handle semantic reasoning and code generation, while deterministic controllers maintain control over execution, metrics, guardrails, and state management.
By Qi Zhang, Yanlin Chen, Wenchao Xiao