arXiv AI
M2K: Making the Model-Kernel Interface Explicit for Reliable CUDA Kernel Verification
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jul 21
KernelBench-Verified: Do LLM-Generated Kernels Actually Beat PyTorch?
arXiv:2607. 16241v1 Announce Type: cross Abstract: Recent large language models (LLMs) can generate custom CUDA kernels that appear to outperform PyTorch on benchmarks such as KernelBench.
arXiv Machine Learning
Jun 11
MPK: A Compiler and Runtime for Mega-Kernelizing Tensor Programs
arXiv:2512. 22219v2 Announce Type: replace-cross Abstract: We introduce Mirage Persistent Kernel (MPK), the first compiler and runtime system that automatically transforms multi-GPU model inference into a single high-performance mega-kernel.
arXiv AI
Jul 29
Kernel Forge: An Agent Harness for LLM-based Generation and Optimization of CUDA Kernels
arXiv:2607. 24762v1 Announce Type: new Abstract: Machine learning models are increasingly embedded in everyday software, and most of their runtime is spent in a small set of compute kernels such as matrix multiplication, convolution, and normalization.
arXiv AI
Aug 13
RealisticTritonBench: A Benchmark for Triton-Kernel Generation in Real-World AI Frameworks
arXiv:2608. 12004v1 Announce Type: cross Abstract: In modern AI frameworks, GPU kernels are key to overall system performance.
arXiv Machine Learning
Sep 23
Accelerating the Mitigation of LLM Inference Nondeterminism Across GPU Architectures
The paper addresses the problem of non‑deterministic outputs from large language models (LLMs) when run on different GPU architectures, caused by floating‑point non‑associativity and hardware‑dependent kernel choices. It proposes a set of fixed‑configuration fused‑upcast GEMM kernels that load 16‑bit weights, upcast to FP32, and perform IEEE‑754 compliant reductions in a problem‑shape‑dependent order, ensuring identical linear‑layer outputs across NVIDIA Ampere, Ada, and Hopper GPUs. The new approach achieves 1.17–3.1× faster end‑to‑end performance than existing solutions and halves weight‑memory traffic while maintaining cross‑architecture reproducibility.
arXiv AI
Sep 2
CUDA-Harness: Harnessing Agentic CUDA Kernel Generation and Optimization from Natural Language
CUDA‑Harness is a framework that enables the generation and optimization of CUDA kernels directly from natural language. It introduces Intermediate‑Structured Generation to bridge high‑level semantics with low‑level kernel code, uses Synthesis‑Based Verification to mitigate reward hacking by providing isolated test data, and employs Feedback‑Adaptive Evolution to prioritize correctness while improving performance. Experiments show the approach generalizes across different large language models, hardware platforms, and even supports C‑to‑CUDA transpilation.