The paper introduces two lightweight inference harnesses, OPTICYCLE and OPTISTATE, designed to enable large language models to translate natural-language optimization problems into solver programs without additional training. Evaluated across nine benchmarks and 11 Qwen variants (0.6B–27B parameters), the harnesses consistently improve macro accuracy, with execution-guided revision boosting performance by up to 49.46 percentage points. Despite their simplicity, both harnesses outperform six adapted agentic workflows and four post-trained optimization models of comparable size, though the benefit of explicit formulation state varies by model.
RSI-Opt introduces Recursive Self-Improvement Optimizer Agents that evolve their own training policies by accumulating knowledge from previous runs, including prompts, playbooks, optimizer code, and decision cadences. A meta‑agent rewrites and archives these policies, enabling reusable procedures and code that adapt to new tasks. Across multiple generations and tasks, RSI‑Opt improves optimizer performance, achieving higher z‑scores on evolution suites, better cross‑model transfer, and significant error reductions on PDE benchmarks compared to fixed‑prompt baselines.
By Binghang Lu, Yige Sun, Guang Lin
VISA (Visual Instruction Synthesis Agent) is an agentic framework that transforms multimodal instruction synthesis into a self‑evolving loop. Each cycle analyzes images to filter constraints, samples new constraint sets, generates candidate instructions, and verifies them using executable tools and large language model judges. Failed samples trigger diagnostic recovery, while accepted samples are evaluated against the target model to estimate difficulty, with all feedback written back to memory to adapt future rounds and provide reward signals for reinforcement learning.
By Min Zeng, Guanxin Tan, Libin Cen, Yawei Wen, Rui Hu, Liuyang Bian, Xiaolong Chen, Xiaoxin Chen
The paper introduces HOW TO TRAIN YOUR LITTLE DRAGON, a method for teaching a small language model to generate and refine GPU optimization scripts. Using a compact, lossless representation of compiler traces, the authors train the model on 108,594 examples derived from TVM MetaSchedule, first with supervised fine‑tuning and then with reinforcement learning that rewards correctness and performance. After training, an 8‑billion‑parameter model achieves over 91% correctness in both construction and refinement tasks and outperforms several much larger LLMs on kernel performance.
By Annabelle Sujun Tang, Wei Tang, Songyao Jiang, Tianhui Li, Shengzhe Xu, Jiin Woo, Kaan Ozkara, Youngsuk Park, Zhen Jia, Hadi Esmaeilzadeh
arXiv:2606. 26758v1 Announce Type: new Abstract: High-performance GPU kernels are critical for reducing the exponentially growing computational costs of large language models (LLMs), but their development heavily relies on manual tuning by domain experts.
By Yaochen Han, Ke Fan, Hongxu Jiang, Wanqi Xu, Weiyu Xie, Runhua Zhang, Chenhui Zhu, Yixiang Zhang
arXiv:2609.08944v1 Announce Type: new
Abstract: Agent skills provide a lightweight way to equip frozen language-model agents with domain knowledge and procedural guidance, yet obtaining high-quality...
By Gaoyuan Li, Meihao Fan, Yizhe Liu, Shaolei Zhang, Ju Fan, Siyi Wang, Jiaheng Hou, Xudong Weng, Honghan Tian, Zang Li
arXiv:2606. 04261v1 Announce Type: new Abstract: Curating training data is among the most consequential yet labor-intensive parts of modern AI development: practitioners iteratively propose, implement, evaluate, and revise data policies against noisy benchmark feedback.
By Feiyang Kang, Hanze Li, Adam Nguyen, Mahavir Dabas, Jiaqi W. Ma, Frederic Sala, Dawn Song, Ruoxi Jia
arXiv:2508. 12551v2 Announce Type: replace-cross Abstract: Linux kernel tuning is essential for optimizing operating system (OS) performance, yet remains challenging due to the complex kernel space, sparse performance feedback, and strong workload sensitivity.
By Hongyu Lin, Yuchen Li, Haoran Luo, Zhenghong Lin, Libo Zhang, Mingjie Xing, Yanjun Wu
arXiv:2607. 29626v1 Announce Type: new Abstract: As LLMs evolve from code completion systems into autonomous scientific agents, evaluating their ability to conduct experiments has become increasingly important.
By Tianyu Huai, Tingshuo Fan, Xinchi Chen, Yining Zheng, Yuxin Wang, Shuang Chen, Jie Zhou, Xuanjing Huang
FlashNext‑RL is a model‑specific reinforcement learning training framework designed for the Qwen3.8‑Flash‑Next language model. It optimizes BF16 training of the full 48‑layer model on a single 8×B200 node by freezing early layers, offloading host memory, and tuning kernels for mixture‑of‑experts, sparse attention, and gated residual streams. The optimizations raise non‑padding training throughput from ~395 tokens/s/GPU to 4,438 tokens/s/GPU and enable a live software‑engineering RL run with 1,024 rollouts and a 131,072‑token context, completing iterations in about 22.4 minutes.
By Zhewei Yao, Yuxiong He
arXiv:2607. 20486v1 Announce Type: new Abstract: Designing optimizers for modern deep learning remains a challenging scientific problem, requiring the joint consideration of optimization geometry, state dynamics, numerical stability, implementation constraints, and empirical generalization.
By Zhongzheng Li, Tiancan Feng, Wenhao Li, Qingsong Ran, Shikun Feng, Xiaoyuan Zhang, Yue Wang, Xiaoguang Zhao
arXiv:2606. 18284v1 Announce Type: cross Abstract: The limiting resource for training agents via reinforcement learning (RL) is increasingly frontier task supply: valid, solvable tasks just difficult enough to train the current model.
By Lorenz Wolf, Connor Watts, Roger Creus Castanyer, Geoffrey Bradway, Maxwill Lin, Augustine N. Mavor-Parker, Matthew Daborn-Sargent