arXiv AI
Jun 9

MEnvAgent: Scalable Polyglot Environment Construction for Verifiable Software Engineering

arXiv:2601. 22859v3 Announce Type: replace-cross Abstract: The evolution of Large Language Model (LLM) agents for software engineering (SWE) is constrained by the scarcity of verifiable datasets, a bottleneck stemming from the complexity of constructing executable environments across diverse languages.

By Chuanzhe Guo, Jingjing Wu, Sijun He, Yang Chen, Zhaoqi Kuang, Shilong Fan, Bingjin Chen, Siqi Bao, Jing Liu, Hua Wu, Qingfu Zhu, Wanxiang Che, Haifeng Wang
arXiv AI
Oct 1

EngramBench: A Capability-Grounded Benchmark for Skill-Evolution Harnesses

EngramBench is a new benchmark designed to evaluate skill evolution in autonomous agents by focusing on genuine capability abstraction rather than solution copying. It includes 30 learning tasks and 13 unseen transfer tasks that require agents to manage complex, multi-hour development cycles with LLM‑simulated users. The study shows that while static skill banks cannot eliminate the need for precise code implementation, they effectively reduce redundant context and cut overall coding time by more than 55%.

By Zhixuan Tan, Pengjie Gu, Zhao Li, Yihan Hu, Xu He, Dong Li, Jianye Hao