arXiv AI
CIVER: Causal Intervention Verification for Reliable Code Review
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Sep 4
SWE-Gate: Passing Functional Tests Is Not Enough for Software Engineering Agents
SWE‑Gate is a new repository‑level benchmark that evaluates software engineering agents on both functional correctness and review‑derived acceptance constraints. It creates 303 repair instances from real pull‑request review comments across 75 open‑source Python projects, providing separate functional and constraint tests along with compliant and non‑compliant patches. Experiments with four LLM backends show that while 644 repairs pass functional tests, 221 fail to meet the review constraints, highlighting a gap between functional success and full repair compliance.
arXiv AI
Jul 7
Is Agentic Code Review Helpful? Mining Developers' Feedback to CodeRabbit Reviews in the Wild
arXiv:2607. 03316v1 Announce Type: cross Abstract: Agentic code review, where autonomous agents provide code review comments on pull requests, is increasingly integrated into development workflows, yet there is limited empirical evidence on how developers respond to such comments in practice.
arXiv AI
Jul 13
The Patchwork Problem in LLM-Generated Code
arXiv:2607. 08981v1 Announce Type: cross Abstract: LLM-generated code often compiles, passes tests, and appears correct, yet breaks once deployed.
arXiv AI
Sep 1
Understanding Automated Program Repair Agents Through the Lens of Traceability: An Empirical Study
The paper presents a systematic analysis of five state‑of‑the‑art automated program repair agents, tracing their decision‑making across 500 real‑world repair tasks. It finds that while the agents perform well on simple fixes, they struggle with logic‑intensive bugs, often producing verbose, overfitted patches that pass tests without addressing root causes. Key bottlenecks identified include poor test generation, limited regression test selection, and reliance on primitive tooling without access to debuggers or advanced program analysis tools.
arXiv AI
Aug 28
From Static to Dynamic: Benchmarking Real-World Code Review with MCR-Bench
The paper introduces MCR-Bench, a benchmark for realistic multi‑round code review that includes 2,269 real‑world tasks across five programming languages, each annotated with fine‑grained defect information and dynamic state labels. Experiments with mainstream large language models show limited overall performance, especially as interaction rounds increase, and reveal that model accuracy varies by defect type and severity. Error analysis identifies key failure mechanisms such as cross‑round temporal misalignment and insufficient long‑range memory.
arXiv AI
Jul 29
How Do LLMs Read Bug Reports? An Empirical Study of Attention in LLMs for Automated Program Repair
arXiv:2607. 25873v1 Announce Type: cross Abstract: Large Language Model (LLM)-based Automated Program Repair systems are advancing rapidly, yet their performance remains inconsistent.