arXiv Machine Learning

MetaLint: Easy-to-Hard Generalization for Code Linting

arXiv:2507. 11687v5 Announce Type: replace-cross Abstract: Large language models excel at code generation but struggle with code linting, particularly in generalizing to unseen or evolving best practices beyond those observed during training.

arXiv AI

Jun 16

FasterPy: An LLM-based Code Execution Efficiency Optimization Framework

arXiv:2512. 22827v2 Announce Type: replace-cross Abstract: Code often suffers from performance bugs.

By Yue Wu, Minghao Han, Ruiyin Li, Peng Liang, Amjed Tahir, Zengyang Li, Qiong Feng, Mojtaba Shahin

llms rag fine-tuning benchmarks

arXiv AI

Aug 3

RePaCA: Leveraging Reasoning Large Language Models for Static Automated Patch Correctness Assessment

arXiv:2507. 22580v2 Announce Type: replace-cross Abstract: Automated Program Repair (APR) seeks to automatically correct software bugs without requiring human intervention.

By Marcos Fuster-Pena, David de-Fitero-Dominguez, Antonio Garcia-Cabot, Eva Garcia-Lopez

llms reinforcement-learning benchmarks

arXiv Machine Learning

Jun 10

CodeAlchemy: Synthetic Code Rewriting at Scale

arXiv:2606. 10087v1 Announce Type: cross Abstract: Pre-training on raw code teaches syntax but provides sparse signal for diverse real-world task formats.

By Ankit Gupta, Aditya Prasad, Rameswar Panda

llms benchmarks

arXiv AI

5d ago

CangjieBench: Benchmarking LLMs on a Low-Resource General-Purpose Programming Language

arXiv:2603. 14501v2 Announce Type: replace-cross Abstract: Large Language Models excel in high-resource programming languages but struggle with low-resource ones.

By Junhang Cheng, Fang Liu, Jia Li, Chengru Wu, Nanxiang Jiang, Li Zhang

llms rag agents benchmarks

arXiv Machine Learning

Jul 22

Spaghetti Architect: A Contamination-Resistant, By-Construction-Labelled, Multi-Language Code Dataset Generator

arXiv:2607. 18642v1 Announce Type: new Abstract: Mined code corpora are abundant but uncontrolled: a snippet's semantics, surface "messiness," and difficulty are whatever the wild contained; there is no known-optimal reference to grade against; and any public sample may already sit in a model's training set.

By Yuxiang Ji

arXiv Machine Learning

Aug 6

From Brute Force to Semantic Insight: Performance-Guided Data Transformation Design with LLMs

arXiv:2601. 03808v2 Announce Type: replace-cross Abstract: Large language models (LLMs) have achieved notable performance in code synthesis; however, data-aware augmentation remains a limiting factor, handled via heuristic design or brute-force approaches.

By Usha Shrestha, Dmitry Ignatov, Radu Timofte

llms reinforcement-learning fine-tuning benchmarks safety

arXiv AI

6d ago

Program Semantic Inequivalence Game with Large Language Models

arXiv:2505. 03818v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) can achieve strong performance on everyday coding tasks, but they can fail on complex tasks that require non-trivial reasoning about program semantics.

By Antonio Valerio Miceli-Barone, Vaishak Belle, Ali Payani

llms agents fine-tuning benchmarks

arXiv AI

Aug 11

Contamination Means Overestimation? A Fine-Grained Empirical Study in Code Intelligence

arXiv:2506. 02791v4 Announce Type: replace-cross Abstract: In recent years, code intelligence has gained increasing importance in the field of automated software engineering.

By Zhen Yang, Hongyi Lin, Yifan He, Junqi Wang, Zeyu Sun, Shuo Liu, Jie Xu, Pengpeng Wang, Zhongxing Yu, Qingyuan Liang

llms nlp fine-tuning

arXiv Machine Learning

Aug 4

Syntax Without Semantics: Teaching Large Language Models to Code in an Unseen Language

arXiv:2605. 15607v2 Announce Type: replace-cross Abstract: Large language models (LLMs) achieve high pass rates on code generation benchmarks, yet whether they can transfer this ability to languages absent from pretraining remains poorly understood.

By Vinayshekhar Bannihatti Kumar, Disha Makhija, Manoj Ghuhan Arivazhagan, Rashmi Gangadharaiah

llms fine-tuning benchmarks

arXiv Machine Learning

Jun 3

Synthetic Hallucinations, Real Gains: Hard Negatives from Frontier Models for FIM Hallucination Mitigation

arXiv:2606. 03130v1 Announce Type: new Abstract: Small open-source code models that power IDE autocomplete still emit hallucinated Fill-in-the-Middle (FIM) completions: syntactically natural calls to methods, parameters, variables, and imports that do not exist in the surrounding project.

By Mahdi Erfanian, Nelson Daniel Troncoso, Aashna Garg, Amabel Gale, Xiaoyu Liu, Pareesa Ameneh Golnari, Shengyu Fu

llms reinforcement-learning fine-tuning benchmarks safety

arXiv AI

Jun 2

Uncovering Competency Gaps in Large Language Models and Their Benchmarks

arXiv:2512. 20638v2 Announce Type: replace-cross Abstract: The evaluation of large language models relies heavily on standardized benchmarks.

By Maty Bohacek, Nino Scherrer, Nicholas Dufour, Thomas Leung, Christoph Bregler, Stephanie C. Y. Chan

llms benchmarks

arXiv Machine Learning

Jun 25

The Generalization Spectrum: A Chromatographic Approach to Evaluating Learning Algorithms

arXiv:2606. 25450v1 Announce Type: new Abstract: Traditional evaluations measure a learning algorithm's final performance on an i.

By Jinghan Zhang, Zerui Cheng, Shiqi Chen, Ge Zhang, Wenhao Huang, Jiashuo Liu, Junxian He, Tianle Cai

reinforcement-learning fine-tuning efficiency benchmarks

Related stories