arXiv AI
Aug 11

A Unified Issue Resolution Benchmark for Requirement Clarification, Planning, and Code Generation for Coding Agents

arXiv:2608. 09072v1 Announce Type: cross Abstract: Large language model-powered coding agents are increasingly used to modify existing code repositories, for example, by adding features or fixing bugs.

By Xin Zhou, Chun Yong Chong, Kisub Kim, Yun Peng, Rui Shu, Zihan Wu, Xu Han, Guowen Yuan, Zeyang Zhuang, Jounghoon Kim, Jeongjin Ju, Seongmin Ju, Taein Yoon, David Lo
arXiv AI
1d ago

E2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch

E2E-SWE is a benchmark that tests large language models’ ability to create complete, functional software repositories from scratch. It includes 186 tasks across 11 programming languages, each requiring an agent to build an installable project based solely on a natural‑language specification and an empty workspace, while passing a hidden test suite. The benchmark was crafted by software engineers and LLMs, then refined through iterative verification by autonomous agents to ensure clarity and solvability.

By Hantian Ding, Chloe Bi, Jiacheng Zhu, John Yang, Matt Deitke, Pengcheng Yin, Zijian Wang, Rui Hou
arXiv AI
Aug 10

Online Monitoring and Corrective Steering of Programming Agents

arXiv:2608. 06701v1 Announce Type: cross Abstract: Fixing GitHub issues in large-scale projects is a long-horizon task, especially when a fix requires changes across multiple locations or the issue description lacks the information needed to localize and repair it.

By Shuyang Liu, Saman Dehghan, Ji Young Kim, Jatin Ganhotra, Martin Hirzel, Reyhaneh Jabbarvand
Hugging Face Trending Papers
Aug 19

SkillForge: Self-Distilling Agents for Project-Specific Issue Resolution

SkillForge is a self‑distillation framework that proactively builds project‑specific knowledge for large language model agents by synthesizing and resolving artificial issues derived from a repository’s test‑covered core functionalities. Rather than waiting for real issues to reveal knowledge gaps, SkillForge generates these synthetic problems, learns reusable entity‑grounded skills, and associates them with relevant repository entities. Experiments with both open‑source and closed‑source models show that this proactive knowledge acquisition consistently outperforms strong baselines in issue resolution tasks.