arXiv AI
Jun 30
Diff-Based Code Corruption using LLMs for Large-Scale Bugfix Benchmarking
arXiv:2606. 29088v1 Announce Type: cross Abstract: There are various benchmarks to evaluate bugfixing capabilities of Large Language Models.
FLARE: Fine-Grained Diagnostic Feedback for LLM Code Refinement
arXiv:2606. 03852v1 Announce Type: cross Abstract: Large language models often generate code with bugs.
Related stories
arXiv Machine Learning
Sep 15
Introspective Uncertainty Estimation for LLM-Based Code Generation
The thesis explores Introspective Uncertainty Estimation (IUE) for large language models (LLMs) in code generation, aiming to determine whether hidden-state representations can indicate functional correctness at both response and line levels. Using LiveCodeBench and BigCodeBench, the study finds that hidden states provide a strong signal for overall correctness, with static single-token probes performing best, while dynamic strategies offer no consistent advantage. Although line-level fault localization is more challenging, a conditional Top‑K ranking approach remains effective, suggesting a two‑stage workflow that first screens responses for risk and then prioritizes line‑level checks.
arXiv AI
Sep 10
SWE-Test: Benchmarking LLM Vulnerability Discovery via Input Prediction
arXiv:2609.06229v1 Announce Type: cross Abstract: Vulnerability discovery is becoming an important ability of large language model (LLM) agents: agents that silently miss real defects leave critical...
arXiv AI
Jun 2
Taming System Complexity: Demystifying Software Engineering Agents in Diagnosing Linux Kernel Faults
arXiv:2505. 19489v2 Announce Type: replace Abstract: The Linux kernel is a critical system, serving as the foundation for numerous systems.
arXiv AI
Jul 23
PerfAgent: Profiler-Guided Iterative Refinement for Repository-Level Code Optimization
arXiv:2607. 19653v1 Announce Type: cross Abstract: Large language model (LLM) agents now perform well on correctness-oriented repository-level tasks, including SWE-Bench issue resolution and feature implementation in real codebases.
arXiv AI
Aug 28
From Static to Dynamic: Benchmarking Real-World Code Review with MCR-Bench
The paper introduces MCR-Bench, a benchmark for realistic multi‑round code review that includes 2,269 real‑world tasks across five programming languages, each annotated with fine‑grained defect information and dynamic state labels. Experiments with mainstream large language models show limited overall performance, especially as interaction rounds increase, and reveal that model accuracy varies by defect type and severity. Error analysis identifies key failure mechanisms such as cross‑round temporal misalignment and insufficient long‑range memory.
arXiv AI
Jul 28
Evaluating and Mitigating the Misguidance Effect of Buggy Code in LLM-Generated Unit Tests
arXiv:2607. 22883v1 Announce Type: cross Abstract: While Large Language Models (LLMs) show great promise for automating unit test generation, recent studies suggest that the quality of generated tests can be negatively impacted when models are prompted with buggy code.
arXiv Machine Learning
Jun 10
Multi-task LLMs for Bug Classification: Efficient Inference with Auxiliary Decoding Heads
arXiv:2606. 09956v1 Announce Type: cross Abstract: The rapid adoption of LLM-powered code generation has dramatically accelerated software development, yet effective verification methods remain severely underdeveloped.
Hugging Face Trending Papers
Jul 24
Evaluating and Mitigating the Misguidance Effect of Buggy Code in LLM-Generated Unit Tests
While Large Language Models (LLMs) show great promise for automating unit test generation, recent studies suggest that the quality of generated tests can be negatively impacted when models are prompted with buggy code. This paper presents a new metric to quantitatively measure the "misguidance effect," a phenomenon where buggy code steers LLMs toward generating tests that validate its erroneous behavior rather than expose it.
arXiv Machine Learning
Aug 31
The Calls are Coming from Inside the Model: Investigating Probe-based Detection of Tool-Calling Errors in LLMs
arXiv:2608.27750v1 Announce Type: new Abstract: The hidden states of large language models (LLMs) are known to capture rich information relating to model knowledge and behavior that can be hard to ex...
arXiv AI
Jun 24
VeriPilot: An LLM-Powered Verilog Debugging Framework
arXiv:2606. 23759v1 Announce Type: cross Abstract: Verilog debugging remains one of the most time-consuming stages in digital circuit design.
arXiv AI
Jul 2
SWE-Doctor: Guiding Software Engineering Agents with Runtime Diagnosis from Multi-Faceted Bug Reproduction Tests
arXiv:2607. 00990v1 Announce Type: cross Abstract: Large language model (LLM)-based software engineering agents are increasingly developed to resolve software issues by generating patches from issue reports and code repositories.