arXiv AI
PROBE: Frontier Coding Agents Find Different Bugs Than Maintainers Fix
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
2d ago
Can Agentic Code Review Catch Vulnerabilities Before They Are Merged?
The paper introduces SecPRBench, a benchmark that evaluates large language model agents on real-world bug‑introducing commits from critical software systems such as Firefox, V8, and OpenSSL. It automates harvesting, building, and validating these commits, and uses an LLM‑as‑a‑judge to match findings with ground‑truth vulnerabilities. Across eight state‑of‑the‑art models, recall ranges from 32.5% to 78.1%, with Grok 4.6 and GPT‑6 sol performing best, while security‑focused harnesses like OpenAI’s Codex Security and Anthropic’s Defending Code influence recall differently.
arXiv AI
Aug 6
Active-SWE: Benchmarking Coding Agents for Proactive Bug Fixing without Issue Reports
arXiv:2608. 04682v1 Announce Type: cross Abstract: Coding agents powered by large language models (LLMs) are increasingly adopted in software engineering (SWE) scenarios, capable of fixing a specific bug in large-scale codebase.
arXiv AI
Jul 14
SWE-MERA: A Dynamic Benchmark for Agenticly Evaluating Large Language Models on Software Engineering Tasks
arXiv:2507. 11059v3 Announce Type: replace-cross Abstract: The rapid advancement of Large Language Models (LLMs) in software engineering has revealed critical limitations in existing benchmarks, particularly the widely used SWE-bench dataset.
arXiv AI
Oct 12
HAFixAgent: History-Aware Agentic Program Repair
HAFixAgent is a history-aware program repair agent that incorporates blame-derived repository context into its repair loop. In a large-scale study on 854 Defects4J and 501 BugsInPy bugs, the agent’s best history heuristic improved repair rates by up to +64.3% with weaker models and outperformed existing agents by significant margins. The three complementary history heuristics together repair more bugs than any single heuristic, with only modest increases in agent steps or tokens.
arXiv AI
Sep 1
Cost-Effective Repository Exploration for Agentic Issue Localization
arXiv:2608.29675v1 Announce Type: cross Abstract: Repository exploration is a distinct and costly stage of coding-agent pipelines: before generating a patch, an agent must identify which repository f...
arXiv AI
Jul 29
How Do LLMs Read Bug Reports? An Empirical Study of Attention in LLMs for Automated Program Repair
arXiv:2607. 25873v1 Announce Type: cross Abstract: Large Language Model (LLM)-based Automated Program Repair systems are advancing rapidly, yet their performance remains inconsistent.