Advanced agents are increasingly demonstrating the potential to operate as autonomous engineers, creating a growing demand for evaluation benchmarks that capture the complexity of real-world development. Such environments typically involve both complex code and large-scale data (i.
arXiv:2606. 15300v1 Announce Type: new Abstract: Advanced agents are increasingly demonstrating the potential to operate as autonomous engineers, creating a growing demand for evaluation benchmarks that capture the complexity of real-world development.
arXiv:2607. 01647v1 Announce Type: cross Abstract: Data science aims to derive actionable insights from heterogeneous raw data, unlocking the value of the massive amounts of data generated in modern society.
Modern coding agents can deliver increasingly large repository-level changes, and recent benchmarks reflect this by emphasizing long-horizon tasks with large reference implementations. Many benchmarks...
arXiv:2609.37143v1 Announce Type: cross Abstract: Modern coding agents can deliver increasingly large repository-level changes, and recent benchmarks reflect this by emphasizing long-horizon tasks wi...
The paper describes progress toward an AI Software Factory that accelerates every stage of the software development lifecycle—Targeting, Coding, Reviewing, and Ops—by generating a metadata exhaust for self‑improvement. It focuses on data systems and evolutionary coding tasks, reporting scaled deployments at Microsoft that achieved three‑fold engineering efficiency over agentic coding and up to 22‑fold token efficiency. The authors also outline several open challenges for further development.
arXiv:2608. 10366v1 Announce Type: new Abstract: Real-world data science involves long-horizon workflows that span data wrangling, exploration, modeling, visualization, and validation, and require coordinated use of tools such as notebooks, IDEs, terminals, browsers, and databases within real operating environments.
arXiv:2511. 06090v3 Announce Type: replace-cross Abstract: Optimizing the performance of large-scale software repositories demands expertise in code reasoning and software engineering (SWE) to reduce runtime while preserving program correctness.
SpecMine is a large-scale corpus that documents Spec-Driven Development (SDD) artifacts in public GitHub repositories. It includes a broad census of 470,795 spec files from 73,030 repositories linked to 17 tools, a focused census of 98,574 Kiro layout files from 12,910 repositories, and a sweep of 5,992 pull requests across 581 repositories that modify specs. The dataset provides enriched metadata, full commit histories, parsed document structures, and over 2.4 million typed references connecting specs to code, sibling documents, PRs, branches, and issues.
arXiv:2606. 11976v1 Announce Type: cross Abstract: Software engineering tools increasingly rely on LLM based agents to localize files to change to resolve a software issue.
Software engineering tools increasingly rely on LLM based agents to localize files to change to resolve a software issue. Most AI agents explore repositories linearly, that is, visiting one directory or file per step.
arXiv:2607. 16845v1 Announce Type: new Abstract: Scientists at European XFEL conduct experiments that generate very large and complex datasets.
arXiv:2603. 03589v3 Announce Type: replace-cross Abstract: Recent advances in large language models (LLMs) transform how machine learning (ML) pipelines are developed and evaluated.