arXiv AI
Jun 12
Toward Instructions-as-Code: Understanding the Impact of Instruction Files on Agentic Pull Requests
arXiv:2606. 13449v1 Announce Type: cross Abstract: AI-agents (e.
Conceptual integrity and counting lines of code
Last week I recorded an episode of the Talking Postgres podcast with Claire Giordano on the subject of "How AI is changing software development". We had a really great conversation.
Related stories
arXiv AI
Sep 7
Beyond Code Generation: Reliability, Verification, and Cost Economics in the Agentic Software Development Lifecycle
The paper examines how AI coding agents are evolving beyond simple autocomplete to perform complex tasks such as repository inspection, multi-file editing, tool execution, test writing, pull request creation, and long-duration work with minimal supervision. It highlights that while these agents boost coding activity, significant bottlenecks remain in review, integration, testing, security, deployment, and production operations, and that the economics of software development are shifting toward variable token, tool, sandbox, CI, and rework costs. The authors synthesize recent research and industry data to propose four engineering concepts—Agentic SDLC Throughput Paradox, Production-Qualified Change, Verification Tax, and an Agentic SDLC Control Plane—to guide the allocation of autonomy within cost, reliability, and human-attention constraints, ultimately reframing the research focus to production-qualified value per dollar, reviewer-hour, and operational risk.
arXiv AI
Oct 15
Reels-RSI: Program-Level Self-Improvement for Code-Rendered Explainer Videos
arXiv:2610.14495v1 Announce Type: new Abstract: Explainer videos -- a worked geometry problem, why the sky is blue, how a dynasty fell -- are usually one more target for pixel video generation. We ar...
arXiv AI
Jul 3
Reasoning effort, not tool access, buys first-try reliability in agentic code generation: an observational study
arXiv:2607. 02436v1 Announce Type: cross Abstract: Agentic coding assistants are increasingly given extra capabilities, such as browser based testing tools and design oriented system prompts, on the assumption that more capability yields better software.
Simon Willison
Sep 11
Feeling sad about AI
Simon Willison reflects on the emotional impact of AI tools that can produce code quickly, noting that many developers experience an initial sense of disheartenment. He argues that recognizing the shift from coding to higher‑level problem solving allows experienced engineers to leverage new tools and add greater value. Willison emphasizes that software engineering has always faced rapid change, so adapting to AI is part of the profession’s ongoing evolution.
Hugging Face Trending Papers
Oct 22
Superintelligent Software Engineering: Governance That Scales with Capability
Software agents are being trained to exceed human engineers; we argue that capability alone does not produce better software engineering. We define superintelligent software engineering as the discipl...
arXiv AI
Nov 3
Superintelligent Software Engineering: Governance That Scales with Capability
The paper defines superintelligent software engineering as building and evolving software with agents whose capability exceeds that of human engineers, emphasizing that governance must scale with capability. It argues that correctness depends on decisions and contracts beyond code, that knowledge decays, and that when agents outpace reviewers, stale knowledge grows. The authors introduce OSTEON, a governance framework that tracks decisions, contracts, and knowledge decay, and report no measured effect of the system while outlining questions that could falsify their thesis.
arXiv AI
Oct 28
Does Your Generalist Skill Optimizer Really Incentivize Agentic Tasks? Rethinking Skill Optimization via Bipartite Attribution
The paper introduces BiPA, a method for evaluating skill optimizers that rewrite natural‑language procedures for agents. By replaying the optimizer’s acceptance rule on identical candidates and recording episodes as bipartite graphs of turns and read text, BiPA isolates the effect of selection from the cost of added input tokens. Experiments on three optimizers, three benchmarks, and two models show that learned skills perform only marginally better than hand‑written seeds, and that apparent dev‑set gains often disappear on test, revealing that optimizers tend to add redundant rules that increase token usage.
arXiv AI
Sep 7
$\tau^\tau$-Bench: An Environment for End-To-End, Realistic Agent Construction
The paper introduces $ au^ au$-Bench, a benchmark that turns the construction of AI agents into a measurable task. In this environment a developer agent receives real business records, client requirements, a production API, an existing codebase, and constraints on cost and models, and must deliver a complete customer‑service agent. The benchmark evaluates performance by deploying the agent against simulated users, revealing that current state‑of‑the‑art models achieve only 23.9% success while an expert‑written reference scores 82.2%.
Hugging Face Trending Papers
Oct 27
Does Your Generalist Skill Optimizer Really Incentivize Agentic Tasks? Rethinking Skill Optimization via Bipartite Attribution
The paper introduces BiPA, a method to evaluate skill optimizers that rewrite natural‑language procedures for agents performing long‑horizon tasks. BiPA replays the optimizer’s acceptance rule on identical candidates to isolate selection effects and records each episode as a bipartite graph of turns and read text, revealing that learned skills only marginally outperform hand‑written seeds and that many gains vanish on test data. The study finds optimizers add extra input tokens and turns, and proposes four design principles for more reliable skill editing.
Hugging Face Trending Papers
Oct 26
kw-bench: Breaking Frontier Harnesses on Real-World Tasks
Each new generation of frontier model pushes the edge of what is possible. Inside a harness -- a loop, a shell, a filesystem, the ability to write and run its own code -- a model can take on increasin...
arXiv AI
Oct 28
kw-bench: Breaking Frontier Harnesses on Real-World Tasks
The paper introduces kw-bench, a benchmark that evaluates coding-agent harnesses on investigative knowledge work, specifically a five‑year VAT reconstruction task across multiple SAP systems. It contrasts generative benchmarks with this real‑world, evidence‑heavy task, showing that model‑agnostic harnesses performed well while lab harnesses failed. The study demonstrates how frontier models can dramatically reduce time and cost for complex audit‑style engagements.