arXiv:2606. 30573v1 Announce Type: new Abstract: We introduce SWE-Interact, a new testbed for evaluating coding agents on multi-turn, interactive, user-driven software engineering tasks.
By Mohit Raghavendra, Anisha Gunjal, Aakash Sabharwal, Yunzhong He
arXiv:2601. 03808v2 Announce Type: replace-cross Abstract: Large language models (LLMs) have achieved notable performance in code synthesis; however, data-aware augmentation remains a limiting factor, handled via heuristic design or brute-force approaches.
By Usha Shrestha, Dmitry Ignatov, Radu Timofte
arXiv:2506. 02791v4 Announce Type: replace-cross Abstract: In recent years, code intelligence has gained increasing importance in the field of automated software engineering.
By Zhen Yang, Hongyi Lin, Yifan He, Junqi Wang, Zeyu Sun, Shuo Liu, Jie Xu, Pengpeng Wang, Zhongxing Yu, Qingyuan Liang
arXiv:2507. 22080v2 Announce Type: replace-cross Abstract: Acquiring high-quality instruction-code pairs is essential for training Large Language Models for code generation.
By Qiushi Sun, Jinyang Gong, Lei Li, Qipeng Guo, Fei Yuan
arXiv:2609.37405v1 Announce Type: cross
Abstract: Large language models (LLMs) are increasingly used for software engineering tasks that require understanding existing source code, including behavior...
By Ali Mohammadi Esfahani, Nafiseh Kahani, Samuel A. Ajila
The paper presents EvalAgent, an AI assistant that automates agent evaluation by encoding domain expertise into evaluation skills such as procedural instructions, reusable code, and dynamic API retrieval. EvalAgent constructs a trace-based pipeline that outputs metrics, executable code, and reports, and is evaluated using a new meta-evaluation framework and AgentEvalBench. Results show that EvalAgent improves the Eval@1 metric from 17.5% to 65% and receives 79.5% human expert preference, while ablation studies confirm the importance of evaluation skills.
By Kang Zhou, Sangmin Woo, Haibo Ding, Kiran Ramnath, Subramanian Chidambaram, Aosong Feng, Vinayak Arannil, Muhyun Kim, Ishan Singh, Darren Wang, Zhichao Xu, Megha Gandhi, Nirmal Prabhu, Soumya Smruti Mishra, Smeet Dhakecha, Vivek Singh, Gouri Pandeshwar, Lin Lee Cheong