arXiv Computation and Language
LintEval: Benchmarking LLMs on Linter-Guided File Refactoring
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv AI
Aug 26
REFINE: A Multi-Agent LLM Approach for Evidence-Guided Code Refactoring
REFINE is a tool-agnostic, evidence-aware multi-agent approach that generates Java file-level refactoring candidates by combining static analysis, smell-informed planning, LLM-based transformation, and automated re-analysis. In experiments on 450 Java files from 15 open-source systems, REFINE reduced detected code smells by 68–73% across three LLM configurations, achieving higher median reductions with smaller edits compared to a direct-prompt baseline. However, the tool’s outputs still pose risks such as assert/fail-call changes and public-method removal, requiring compilation, testing, dependency analysis, and human review before deployment.
arXiv Machine Learning
Aug 11
MetaLint: Easy-to-Hard Generalization for Code Linting
arXiv:2507. 11687v5 Announce Type: replace-cross Abstract: Large language models excel at code generation but struggle with code linting, particularly in generalizing to unseen or evolving best practices beyond those observed during training.
arXiv AI
Sep 7
RefactorPlatform: An Open-Source Harness for Controlled Evaluation of Repository-Scale Refactoring Agents
RefactorPlatform is an open‑source harness that standardizes the evaluation of repository‑scale refactoring agents by fixing the environment and systematically varying design choices such as model backbone, execution regime, and prompt specificity. Each run operates in an isolated workspace, logs detailed telemetry, and verifies changes with AST‑based checks. Experiments on 100 RefactorBench tasks show that AST‑aware chunking improves performance by 25‑30%, a lean retrieval‑augmented single agent outperforms a sub‑agent configuration, and retrieval’s accuracy gains offset its token overhead, keeping cost per successful refactoring unchanged.
arXiv AI
Jun 2
scicode-lint: Detecting Methodology Bugs in Scientific Python Code with LLM-Generated Patterns
arXiv:2603. 17893v2 Announce Type: replace-cross Abstract: Methodology bugs in scientific Python code produce plausible but incorrect results that traditional linters and static analysis tools cannot detect.
arXiv AI
Jun 9
CodeTaste: Can LLMs Generate Human-Level Code Refactorings?
arXiv:2603. 04177v2 Announce Type: replace-cross Abstract: LLM coding agents can generate working code, but their solutions often accumulate complexity, duplication, and architectural debt.
arXiv AI
Jun 9
Beyond Pass Rate: A Multilingual, Execution-Grounded Evaluation of Open Code LLMs
arXiv:2606. 08840v1 Announce Type: new Abstract: Code generation models are typically compared using compact execution benchmarks and aggregate pass rates, but such summaries obscure how performance varies across programming languages, problem families, and failure modes.