arXiv AI June 6, 2026 By Le Chen, Nuo Xu, Winson Chen, Bin Lei, Pei-Hung Lin, Dunzhi Zhou, Rajeev Thakur, Caiwen Ding, Ali Jannesari, Chunhua Liao

Beyond Code Pairs: Dialogue-Based Data Generation for LLM Code Translation

Read the original on arXiv AI →

arXiv:2512. 03086v2 Announce Type: replace-cross Abstract: Large language models (LLMs) have shown remarkable capabilities in code translation, yet their performance deteriorates in low-resource programming domains such as Fortran and emerging frameworks like CUDA, where high-quality parallel data are scarce.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

6d ago

ORCA: Evaluating LLMs on Data Science Code Translation

ORCA is a new benchmark for evaluating large language models on Data Science Code Translation (DSCT), comprising two settings: ORCA-MAIN with 1,600 grounding-level tasks across data querying, manipulation, and deep learning, and ORCA-PROJECT with 200 full-project translation tasks across seven data‑science task types. Each task includes reference translations and test cases to verify functional equivalence, and a multi‑stage quality verification process ensures task correctness. Experiments show that even state‑of‑the‑art LLMs perform poorly on DSCT, with Claude‑Opus‑4.6 achieving only 56.92% success on ORCA‑MAIN and 33.67% on ORCA‑PROJECT, while an intent‑augmented approach improves success rates by 4.80% and 5.33% respectively.

By Xiaolong Li, Jinyang Li, Bowen Qin, Ge Qu, Nan Huo, Xiaohan Xu, Shipei Lin, Reynold Cheng

llms robotics benchmarks

arXiv AI

Jul 24

Learn2Zinc: Fine-tuning Small Language Models for Text-to-Model Translation in MiniZinc

arXiv:2607. 20456v1 Announce Type: cross Abstract: Large language models excel at code generation for mainstream programming languages but struggle with rare, domain-specific languages such as MiniZinc, a constraint modeling language for combinatorial problems.

By Serdar Kadioglu, Karthik Uppuluri

llms fine-tuning

arXiv AI

Jul 28

ParBench: A Benchmark for Reliable Evaluation of LLM Parallel Code Translation

arXiv:2607. 22588v1 Announce Type: new Abstract: Modern compute-intensive software must migrate across a changing ecosystem of accelerators, programming APIs, compiler stacks, and portability layers, including CUDA, OpenMP, OpenCL, and OpenMP target offload.

By Samyak Jhaveri, Erel Kaplan, Tom Yotam, Le Chen, Tomer Bitan, Niranjan Hasabnis, Gal Oren

llms agents benchmarks

arXiv Machine Learning

Aug 6

From Brute Force to Semantic Insight: Performance-Guided Data Transformation Design with LLMs

arXiv:2601. 03808v2 Announce Type: replace-cross Abstract: Large language models (LLMs) have achieved notable performance in code synthesis; however, data-aware augmentation remains a limiting factor, handled via heuristic design or brute-force approaches.

By Usha Shrestha, Dmitry Ignatov, Radu Timofte

llms reinforcement-learning fine-tuning benchmarks safety

arXiv AI

Aug 14

CangjieBench: Benchmarking LLMs on a Low-Resource General-Purpose Programming Language

arXiv:2603. 14501v2 Announce Type: replace-cross Abstract: Large Language Models excel in high-resource programming languages but struggle with low-resource ones.

By Junhang Cheng, Fang Liu, Jia Li, Chengru Wu, Nanxiang Jiang, Li Zhang

llms rag agents benchmarks

arXiv AI

Jul 2

LLVM-Bench: Benchmarking and Advancing Large Language Models for LLVM Compiler Issue Resolution

arXiv:2607. 00700v1 Announce Type: cross Abstract: LLVM is a widely used compiler infrastructure whose scale and complexity make issue resolution labor-intensive and challenging.

By Zhao Tian, Yingquan Zhao, Chenyao Suo, Meng Wang, Junjie Chen

llms agents benchmarks

One email a morning, machine-written

Related stories

ORCA: Evaluating LLMs on Data Science Code Translation

Learn2Zinc: Fine-tuning Small Language Models for Text-to-Model Translation in MiniZinc

ParBench: A Benchmark for Reliable Evaluation of LLM Parallel Code Translation

From Brute Force to Semantic Insight: Performance-Guided Data Transformation Design with LLMs

CangjieBench: Benchmarking LLMs on a Low-Resource General-Purpose Programming Language

LLVM-Bench: Benchmarking and Advancing Large Language Models for LLVM Compiler Issue Resolution