arXiv AI By Anshu Raina, Peyman Razaghi, Yuankai Chen, Cheng Yao, Yao Fu, Vidushi Goyal, Devang Patel, Wen Xie, Zhenyu Gu, Emad Barsoum

Beyond the Trial-and-Error Loop: Hybrid Projection and Automated Tuning for Distributed Training

Read the original on arXiv AI →

The paper introduces Primus Projection, a hybrid engine that combines GPU microbenchmarks with analytical scaling to predict multi‑node distributed training performance, and Primus Tuning Agent, an automated search that uses the projection engine as a fast oracle to explore the vast configuration space of parallelism and micro‑batching. Together, they reduce the need for costly trial‑and‑error on hundreds of GPUs, achieving predictions within ~10% of real throughput and enabling significant throughput gains—up to +27% in a Mixtral 8x22B case study—within minutes of single‑node exploration.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Jun 30

KernelSight-LM: A Kernel-Level LLM Inference Simulator

arXiv:2606. 28565v1 Announce Type: cross Abstract: As large language models (LLMs) move into production serving, practitioners must rapidly evaluate inference performance across diverse hardware, models, and serving parameters to meet cost and latency targets.

By Xiteng Yao, Taeho Kim, Hengzhi Pei, Xinle Liu, Kyle Ulrich, Leonard Lausen, Ashish Khetan, Xiang Song, George Karypis, Martin Herbordt
arXiv AI
Sep 11

KernelGenBench: Can LLMs and Agents Write Efficient Kernels Across Operator Sources and Hardware Platforms?

KernelGenBench is a unified benchmark that evaluates large language models and agentic systems for generating efficient Triton kernels across diverse operator sources and hardware platforms. It covers 210 operators from PyTorch ATen, vLLM, and cuBLAS, and tests a 110‑operator subset on six different chips, consuming over 15 billion tokens in evaluation. The study finds that no single method dominates across all sources and platforms, with significant variations in correctness and performance depending on the operator source and hardware, and that agentic approaches require millions of tokens per successful operator.

By Peiyu Zang, Jian Tao, Jialing Zhang, Yichen Yuan, Wentao Zhang, Guang Liu, Yonghua Lin
arXiv AI
Sep 16

Ave: Guiding Agentic GPU Optimization Using Data-Flow Invariants

arXiv:2604.18616v2 Announce Type: replace-cross Abstract: LLM coding agents can generate correct GPU kernels, but their performance still trails expert libraries. Reaching peak throughput requires co...

By Haohui Mai, Xiaoyan Guo, Xiangyun Ding, Daifeng Li, Qiuchu Yu, Chenzhun Guo, Cong Wang, Jiacheng Zhao, Christos Kozyrakis, Binhang Yuan