Hugging Face Trending Papers
SWE-Duel: Towards a Self-Scaling Arena for Coding Agents
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv Computation and Language
1d ago
SWE-Duel: Towards a Self-Scaling Arena for Coding Agents
arXiv:2610.12569v1 Announce Type: new Abstract: Static coding benchmarks are fixed at release. Over time their tasks leak into training data, and stronger agents eventually saturate them. We argue th...
arXiv AI
4d ago
TestJack: Should you trust the results in coding benchmarks? Agentic Coding Benchmarks Auditing via Evaluator Evolution
TestJack is a scalable framework that audits large language model (LLM) coding agents by generating new tests for each trial, focusing on prompt requirements that may be violated. It keeps only the tests passed by the ground‑truth patch and re‑examines failures, providing replayable evidence for each confirmed error. In experiments across six model backends and five benchmarks, TestJack found that 34.4% of trials deemed correct actually violated task requirements, reducing the overall resolution rate from 50.6% to 33.2%.
arXiv AI
Jul 3
TestEvo-Bench: An Executable and Live Benchmark for Test and Code Co-Evolution
arXiv:2607. 02469v1 Announce Type: cross Abstract: Software tests and code evolve together: a code change should be followed by new or updated tests that record the new software behavior.
arXiv AI
Oct 1
Zero2Repo: Can Coding Agents Build Repositories from Scratch?
arXiv:2609.38269v1 Announce Type: cross Abstract: Coding agents are increasingly asked to build software rather than patch it, yet benchmarks for from-scratch repository construction are mostly limit...
arXiv AI
Aug 25
SWE Refactor Bench: Can Coding Agents Complete a Long-Horizon, Whole-Repository Stack Migration?
The paper introduces SWE Refactor Bench, a benchmark of 20 whole‑repository migrations aimed at testing whether coding agents can autonomously perform long‑horizon stack migrations. It employs a three‑stage evaluation—Migration Audit, Behavioural Tests, and Agentic Verification—to assess both migration completeness and behavioural correctness, revealing that only 5.4 % of runs succeed across all stages. The study shows that agents struggle with migration tasks, especially language rewrites, and highlights the need for more robust coding‑agent capabilities.
arXiv Machine Learning
Jul 10
DeepSWE: Measuring Frontier Coding Agents on Original, Long-Horizon Engineering Tasks
arXiv:2607. 07946v1 Announce Type: cross Abstract: DeepSWE is a benchmark of 113 original, long-horizon software engineering tasks for evaluating coding agents.