arXiv Machine Learning

CodeAlchemy: Synthetic Code Rewriting at Scale

arXiv:2606. 10087v1 Announce Type: cross Abstract: Pre-training on raw code teaches syntax but provides sparse signal for diverse real-world task formats.

arXiv AI

Jul 7

Formal Disco: Scalable Open-Ended Generation of Formally Verified Programs

arXiv:2607. 04631v1 Announce Type: new Abstract: The cost of producing code is rapidly diminishing with increasingly capable AI agents, while quality assurance of generated programs has not kept pace.

By Gabriel Poesia, Simon Henniger, Tzu-Han Hsu, Yilun Du, Nada Amin

llms agents fine-tuning efficiency

arXiv AI

6d ago

Program Semantic Inequivalence Game with Large Language Models

arXiv:2505. 03818v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) can achieve strong performance on everyday coding tasks, but they can fail on complex tasks that require non-trivial reasoning about program semantics.

By Antonio Valerio Miceli-Barone, Vaishak Belle, Ali Payani

llms agents fine-tuning benchmarks

arXiv Machine Learning

Jul 10

Selective Left-Shift: Turning Test-Time Compute and Difficulty-based Curation into Training Data for Low-Resource Code Generation

arXiv:2607. 07748v1 Announce Type: new Abstract: Large Language Models achieve strong code generation for high resource languages like Python and Java but suffer sharp performance drops on Low-Resource Programming Languages~(LRPLs) such as Julia.

By Didula Samaraweera, Anjana Supun, Srinath Perera

llms reinforcement-learning fine-tuning benchmarks

arXiv AI

Jul 22

SciCodePile: A 128GB Corpus and Executable Benchmark for Challenging Scientific Code Generation

arXiv:2607. 19104v1 Announce Type: cross Abstract: Large language models (LLMs) excel at general-purpose code generation, yet how well they handle scientific code remains an open question.

By Weifeng Sun, Ye Fan, Yuchen Chen, Gou Tan, Jieke Shi, Yuan Yidi, Swee Liang Wong, Jonathan Pan, David Lo

llms fine-tuning benchmarks

arXiv AI

5d ago

CangjieBench: Benchmarking LLMs on a Low-Resource General-Purpose Programming Language

arXiv:2603. 14501v2 Announce Type: replace-cross Abstract: Large Language Models excel in high-resource programming languages but struggle with low-resource ones.

By Junhang Cheng, Fang Liu, Jia Li, Chengru Wu, Nanxiang Jiang, Li Zhang

llms rag agents benchmarks

arXiv AI

Jun 2

BenchEvolver: Frontier Task Synthesis via Solution-Centric Evolution

arXiv:2606. 01286v1 Announce Type: cross Abstract: The rapid progress of frontier large language models has led to widespread benchmark saturation, limiting the ability of existing datasets to differentiate model capabilities or provide useful training signal.

By Yangzhen Wu, Aaron J. Li, Wenjie Ma, Li Cao, Ziheng Zhou, Mert Cemri, Shu Liu, Yuran Xiu, Chenxiao Yan, Haikun Zhao, Bin Yu, Ion Stoica, Dawn Song

llms reinforcement-learning benchmarks

arXiv AI

Jul 29

Learning from 53.6K Real-World Developer Edits of AI-Generated Code

arXiv:2607. 25130v1 Announce Type: cross Abstract: Imperfections in AI-generated code require that software developers modify the generated code manually, or by re-prompting an AI programming assistant.

By Jenny T. Liang, Mihika Bairathi, Wayne Chi, Ameet Talwalkar, Nishant Subramani, Valerie Chen

llms fine-tuning benchmarks

arXiv AI

Jun 16

Open-SWE-Traces: Advancing Dual-Mode Multilingual Distillation for Software Engineering Agents

arXiv:2606. 16038v1 Announce Type: cross Abstract: The path toward autonomous software engineering is currently bottlenecked by a severe deficit of diverse, large-scale trajectory data.

By Wasi Uddin Ahmad, Nikolai Ludwig, Somshubra Majumdar, Boris Ginsburg

llms agents fine-tuning efficiency

arXiv AI

Jul 28

DataOrchestra: Learning to Orchestrate Per-Example Curation of Pretraining Data

arXiv:2607. 24717v1 Announce Type: cross Abstract: Pretraining data processing is critical to the downstream performance of Large Language Models (LLMs).

By Zhen Huang, Yikun Wang, Shijie Xia, Pengfei Liu

llms benchmarks

arXiv AI

Jun 6

Beyond Code Pairs: Dialogue-Based Data Generation for LLM Code Translation

arXiv:2512. 03086v2 Announce Type: replace-cross Abstract: Large language models (LLMs) have shown remarkable capabilities in code translation, yet their performance deteriorates in low-resource programming domains such as Fortran and emerging frameworks like CUDA, where high-quality parallel data are scarce.

By Le Chen, Nuo Xu, Winson Chen, Bin Lei, Pei-Hung Lin, Dunzhi Zhou, Rajeev Thakur, Caiwen Ding, Ali Jannesari, Chunhua Liao

llms fine-tuning

arXiv AI

Jul 28

CodeEvo: Interaction-Driven Synthesis of Code-centric Data through Hybrid and Iterative Feedback

arXiv:2507. 22080v2 Announce Type: replace-cross Abstract: Acquiring high-quality instruction-code pairs is essential for training Large Language Models for code generation.

By Qiushi Sun, Jinyang Gong, Lei Li, Qipeng Guo, Fei Yuan

llms agents fine-tuning benchmarks

arXiv AI

Jul 28

Loong: Synthesize Long Chain-of-Thoughts at Scale through Verifiers

arXiv:2509. 03059v2 Announce Type: replace-cross Abstract: Recent advances in Large Language Models (LLMs) have shown that their reasoning capabilities can be significantly improved through Reinforcement Learning with Verifiable Reward (RLVR), particularly in domains like mathematics and programming, where ground-truth correctness can be automatically evaluated.

By Xingyue Huang, Rishabh, Gregor Franke, Ziyi Yang, Jiamu Bai, Weijie Bai, Jinhe Bi, Zifeng Ding, Yiqun Duan, Chengyu Fan, Wendong Fan, Xin Gao, Ruohao Guo, Yuan He, Zhuangzhuang He, Xianglong Hu, Neil Johnson, Bowen Li, Fangru Lin, Siyu Lin, Tong Liu, Yunpu Ma, Hao Shen, Hao Sun, Beibei Wang, Fangyijie Wang, Hao Wang, Haoran Wang, Yang Wang, Yifeng Wang, Zhaowei Wang, Ziyang Wang, Yifan Wu, Zikai Xiao, Chengxing Xie, Fan Yang, Junxiao Yang, Qianshuo Ye, Ziyu Ye, Guangtao Zeng, Yuwen Ebony Zhang, Zeyu Zhang, Zihao Zhu, Bernard Ghanem, Philip Torr, Guohao Li

llms agents reinforcement-learning benchmarks

Related stories