arXiv AI

T-LLM Compiler: Trusted LLM-based Code Optimization and Verification Framework

arXiv:2608. 14953v1 Announce Type: new Abstract: Recent advances in Large Language Models (LLMs) have opened opportunities to apply high-level code transformations to the field of code optimization, and it has since emerged as one of the most fundamental tasks for LLMs to perform; however, at present, LLMs struggle to apply wide-ranging code optimization tasks due to both the complexity of the code and the inability to independently verify the correctness of the transformations.

arXiv AI

Aug 3

Agentic Harness for Real-World Compilers

arXiv:2603. 20075v2 Announce Type: replace-cross Abstract: Compilers are critical to modern computing, yet fixing compiler bugs is difficult.

By Yingwei Zheng, Cong Li, Shaohua Li, Yuqun Zhang, Zhendong Su

llms agents benchmarks

arXiv AI

Aug 11

Quokka: Accelerating Program Verification with LLMs via Invariant Synthesis

arXiv:2509. 21629v4 Announce Type: replace-cross Abstract: Program verification relies on loop invariants, yet automatically discovering strong invariants remains a long-standing challenge.

By Anjiang Wei, Tianran Sun, Tarun Suresh, Haoze Wu, Ke Wang, Alex Aiken

llms fine-tuning benchmarks

arXiv AI

Jul 2

LLVM-Bench: Benchmarking and Advancing Large Language Models for LLVM Compiler Issue Resolution

arXiv:2607. 00700v1 Announce Type: cross Abstract: LLVM is a widely used compiler infrastructure whose scale and complexity make issue resolution labor-intensive and challenging.

By Zhao Tian, Yingquan Zhao, Chenyao Suo, Meng Wang, Junjie Chen

llms agents benchmarks

arXiv AI

Jun 19

AutoPass: Evidence-Guided LLM Agents for Compiler Performance Tuning

arXiv:2606. 20373v1 Announce Type: cross Abstract: Large Language Models (LLMs) show promise for code compilation tasks, but applying them to runtime performance tuning is difficult due to complex microarchitectural effects and noisy runtime measurements.

By Zepeng Li, Jie Ren, Zhanyong Tang, Jie Zheng, Zheng Wang

llms agents fine-tuning benchmarks

arXiv AI

Aug 5

Can Large Language Models Recover Semantic Optimization Opportunities That Compilers Miss?

arXiv:2608. 03983v1 Announce Type: cross Abstract: Optimizing compilers miss profitable transformations when their enabling semantics are absent from the analyzed program representation.

By Hailong Jiang, Feng Yu, Emran Hossain, Jianfeng Zhu, Mengfei Ren, Qiang Guan, Chunwei Xia

llms benchmarks

arXiv AI

Jun 16

DualGauge: Automated Joint Security-Functionality Benchmarking of Specification-Only Code Generation by LLMs and Coding Agents

arXiv:2511. 20709v2 Announce Type: replace-cross Abstract: Large language models (LLMs) and LLM-based coding agents are now used to generate code from natural-language specifications, yet ensuring such code is both functionally correct and secure remains a challenge.

By Rupam Patir, Keyan Guo, Suvadra Barua, Abhijeet Pathak, Dinesh Gudimetla, Jiawei Guo, Hongxin Hu, Haipeng Cai

llms agents fine-tuning efficiency benchmarks

arXiv AI

Aug 11

Effect of Abstractions and Prompting Strategies on LLM-Guided High-Performance Optimizations

arXiv:2608. 08085v1 Announce Type: cross Abstract: Code performance optimization is a vital aspect of modern software development, as it enables faster response times and reduced resource usage.

By Ji\v{r}\'i Klepl, Maty'a\v{s} Brabec, Martin Kruli\v{s}

llms benchmarks

arXiv AI

Jul 28

ParBench: A Benchmark for Reliable Evaluation of LLM Parallel Code Translation

arXiv:2607. 22588v1 Announce Type: new Abstract: Modern compute-intensive software must migrate across a changing ecosystem of accelerators, programming APIs, compiler stacks, and portability layers, including CUDA, OpenMP, OpenCL, and OpenMP target offload.

By Samyak Jhaveri, Erel Kaplan, Tom Yotam, Le Chen, Tomer Bitan, Niranjan Hasabnis, Gal Oren

llms agents benchmarks

arXiv AI

Jun 6

Beyond Code Pairs: Dialogue-Based Data Generation for LLM Code Translation

arXiv:2512. 03086v2 Announce Type: replace-cross Abstract: Large language models (LLMs) have shown remarkable capabilities in code translation, yet their performance deteriorates in low-resource programming domains such as Fortran and emerging frameworks like CUDA, where high-quality parallel data are scarce.

By Le Chen, Nuo Xu, Winson Chen, Bin Lei, Pei-Hung Lin, Dunzhi Zhou, Rajeev Thakur, Caiwen Ding, Ali Jannesari, Chunhua Liao

llms fine-tuning

arXiv Machine Learning

Jul 10

Functional and Secure Code Generation with Task Vectors

arXiv:2607. 07881v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used for code generation, but they struggle to generate functional code free of security vulnerabilities.

By Felix Wang, Anudeep Das, Mei Nagappan, N. Asokan

llms benchmarks safety

arXiv AI

Jun 26

An Empirical Study of LLM-Generated Specifications for VeriFast

arXiv:2606. 26490v1 Announce Type: cross Abstract: Static verification tools can assure industrial scale software, but require significant human labor to write specifications.

By Wen Fan, Minh Tran, Sanya Dod, Xin Hu, Marilyn Rego, Danning Xie, Jenna DiVincenzo, Lin Tan

llms

arXiv AI

Aug 11

P$^{3}$: Joint Program-and-Proof Planning for Verified Code Generation

arXiv:2608. 09277v1 Announce Type: new Abstract: Verified code generation asks a large language model (LLM) to generate both an executable program and a machine-checkable proof that the program meets a formal specification, promising software that is correct by construction.

By Zenan Li, Ziran Yang, Peiyang Song, Zhaoyu Li, Kaiyu Yang

llms agents benchmarks

Related stories