arXiv AI October 16, 2026 By Annabelle Sujun Tang, Wei Tang, Songyao Jiang, Tianhui Li, Shengzhe Xu, Jiin Woo, Kaan Ozkara, Youngsuk Park, Zhen Jia, Hadi Esmaeilzadeh

How to Train Your Little Dragon: Teaching a Small Language Model to Construct and Refine GPU Optimization Scripts

Read the original on arXiv AI →

The paper introduces HOW TO TRAIN YOUR LITTLE DRAGON, a method for teaching a small language model to generate and refine GPU optimization scripts. Using a compact, lossless representation of compiler traces, the authors train the model on 108,594 examples derived from TVM MetaSchedule, first with supervised fine‑tuning and then with reinforcement learning that rewards correctness and performance. After training, an 8‑billion‑parameter model achieves over 91% correctness in both construction and refinement tasks and outperforms several much larger LLMs on kernel performance.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Machine Learning

Jul 3

DecompRL: Solving Harder Problems by Learning Modular Code Generation

arXiv:2607. 02390v1 Announce Type: new Abstract: How can Large Language Models (LLMs) solve problems they currently cannot?

By Juliette Decugis, Fabian Gloeckle, Francis Bach, Taco Cohen, Gabriel Synnaeve

llms reinforcement-learning

arXiv Machine Learning

Jul 10

Selective Left-Shift: Turning Test-Time Compute and Difficulty-based Curation into Training Data for Low-Resource Code Generation

arXiv:2607. 07748v1 Announce Type: new Abstract: Large Language Models achieve strong code generation for high resource languages like Python and Java but suffer sharp performance drops on Low-Resource Programming Languages~(LRPLs) such as Julia.

By Didula Samaraweera, Anjana Supun, Srinath Perera

llms reinforcement-learning fine-tuning benchmarks

arXiv Machine Learning

Jul 7

NKI-Agent: Domain-Specific Fine-Tuning and Agentic Tool Use for Neuron Kernel Generation

arXiv:2607. 04395v1 Announce Type: new Abstract: Recent agentic approaches to LLM-based kernel generation have achieved impressive results on CUDA.

By Junjie Tang, Jun Huan, Hao Zhou, Yuhao Zhang, Lin Wang

llms agents reinforcement-learning fine-tuning benchmarks

arXiv AI

Jul 24

Multi-turn RL with Structural and Performance Aware Rewards for CUDA Kernel Generation

arXiv:2607. 20908v1 Announce Type: cross Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a powerful technique to enhance the reasoning capacity of LLMs for optimized code generation.

By Quazi Ishtiaque Mahmud, Nesreen K. Ahmed, Ali Jannesari

llms agents reinforcement-learning benchmarks

arXiv AI

Aug 5

On the Limits of Layer Pruning for Generative Reasoning in Large Language Models

arXiv:2602. 01997v3 Announce Type: replace-cross Abstract: Recent work has shown that layer pruning can effectively compress large language models (LLMs) while retaining strong performance on classification benchmarks, often with little or no finetuning.

By Safal Shrestha, Anubhav Shrestha, Minwu Kim, Aadim Nepal, Keith Ross

llms fine-tuning efficiency benchmarks

arXiv Machine Learning

Aug 4

LEAP: Lean Environment-Feedback via Adaptive Pruning for Code RL in GPU Kernel Generation

arXiv:2608. 01804v1 Announce Type: new Abstract: Post-training large language models (LLMs) via reinforcement learning (RL) has significantly advanced code generation capabilities.

By Tankun Li, Zhi Chen, Yaohua Tang

llms reinforcement-learning efficiency benchmarks safety

One email a morning, machine-written

Related stories

DecompRL: Solving Harder Problems by Learning Modular Code Generation

Selective Left-Shift: Turning Test-Time Compute and Difficulty-based Curation into Training Data for Low-Resource Code Generation

NKI-Agent: Domain-Specific Fine-Tuning and Agentic Tool Use for Neuron Kernel Generation

Multi-turn RL with Structural and Performance Aware Rewards for CUDA Kernel Generation

On the Limits of Layer Pruning for Generative Reasoning in Large Language Models

LEAP: Lean Environment-Feedback via Adaptive Pruning for Code RL in GPU Kernel Generation