FastKernels is a benchmark comprising 384 GPU kernel tasks from 47 architectures across 8 categories, designed to evaluate kernel generation agents in realistic production settings. It covers 94.6% of HuggingFace Transformers architectures, scoring kernels on correctness, coverage, and speedup within their native execution paths. The benchmark reveals that kernel‑level speedups often collapse to modest end‑to‑end gains and that isolated performance rankings can misrepresent real‑world effectiveness.
arXiv:2607. 14541v1 Announce Type: new Abstract: Existing GPU kernel generation benchmarks draw problems from synthetic or curated sources that diverge from deployed workloads.
KernelOPT is a multi‑agent system that optimizes GPU kernels generated by compilers like PyTorch Inductor by treating compiled models as structured artifacts. It preserves vendor library calls and focuses on Triton sub‑kernels, using five profiling‑guided LLM agents and a four‑gate verification cascade to ensure correctness and performance before re‑stitching the model. On 250 KernelBench problems, KernelOPT achieves geometric mean speedups of 1.40×, 1.15×, and 1.07× over torch.compile at three optimization levels.
arXiv:2609.21058v1 Announce Type: cross Abstract: Language models can now write GPU kernels that outperform PyTorch. We evaluate five model configurations on KernelBench level 1 and find that a front...
KernelOPT is a multi-agent system that optimizes GPU kernels generated by compilers like PyTorch Inductor by treating compiled models as structured artifacts. It preserves vendor library calls and focuses on Triton sub-kernels, using five profiling-guided LLM agents and a four-gate verification cascade to filter and validate candidates. On 250 KernelBench problems, KernelOPT achieves geometric mean speedups of 1.40×, 1.15×, and 1.07× over torch.compile at different optimization levels.
KernelGenBench is a unified benchmark that evaluates large language models and agentic systems for generating efficient Triton kernels across diverse operator sources and hardware platforms. It covers 210 operators from PyTorch ATen, vLLM, and cuBLAS, and tests a 110‑operator subset on six different chips, consuming over 15 billion tokens in evaluation. The study finds that no single method dominates across all sources and platforms, with significant variations in correctness and performance depending on the operator source and hardware, and that agentic approaches require millions of tokens per successful operator.
arXiv:2604. 01489v2 Announce Type: replace Abstract: High-performance GPU kernels are critical to modern machine learning systems, yet developing them remains a manual, expert-driven process.
arXiv:2607. 27231v1 Announce Type: cross Abstract: Large language models (LLMs) have significantly increased the demand for efficient accelerator kernels, but kernel development remains a highly specialized and labor-intensive task.
arXiv:2606. 02963v1 Announce Type: new Abstract: Production inference increasingly targets a heterogeneous mix of accelerators.
arXiv:2605. 09708v2 Announce Type: replace-cross Abstract: We present Metal-Sci, a 10-task benchmark of scientific Apple Silicon Metal compute kernels spanning six optimization regimes (stencils, all-pairs in $n$-body problems, multi-field Boltzmann, neighbor-list molecular dynamics, multi-kernel PDE, FFT).
arXiv:2606. 28565v1 Announce Type: cross Abstract: As large language models (LLMs) move into production serving, practitioners must rapidly evaluate inference performance across diverse hardware, models, and serving parameters to meet cost and latency targets.
arXiv:2608. 12629v1 Announce Type: new Abstract: GPU kernel agents and GPU programming languages have advanced separately, leaving expert kernels difficult to reproduce.
arXiv:2607. 24762v1 Announce Type: new Abstract: Machine learning models are increasingly embedded in everyday software, and most of their runtime is spent in a small set of compute kernels such as matrix multiplication, convolution, and normalization.