arXiv Machine Learning By Javid Baydamirli, Ismayil Ismayilov, Kaan Oktay, Didem Unat

Compiling Specialized GPU-Initiated Networking

Read the original on arXiv Machine Learning →

Obaleth is a compiler that incorporates GPU-initiated communication into a kernel’s intermediate representation, enabling co‑optimization of communication and computation. It generates specialized transport code for NVIDIA and AMD GPUs, optimizing buffer lifetimes, memory ordering, synchronization, and batching. Benchmarks show significant speedups: up to 3.3× faster small‑message sends on H100 GPUs, 1.25×–2.3× faster all‑to‑all on 256 GPUs compared to NVSHMEM and NCCL GIN, and 1.29× faster Qwen3‑235B inference on 16 B200 GPUs in vLLM.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning
Sep 30

Mixture-of-Kittens: MoE Megakernel for NVL72s

arXiv:2609.36070v1 Announce Type: cross Abstract: AI accelerator systems are rapidly consolidating into scale-up architectures, where tens to thousands of GPUs communicate over high-bandwidth, single...

By Stuart H. Sul, Nash Brown, Henry Wildermuth, William Lin, Federico Cassano, Christopher R\'e
arXiv Machine Learning
1d ago

HetCCL: Efficient LLM Training on Heterogeneous Vendor GPUs

arXiv:2601.22585v2 Announce Type: replace-cross Abstract: The rapid growth of large language models is driving organizations to expand their GPU clusters, often with GPUs from multiple vendors. Howev...

By Heehoon Kim, Chris Jaehwan Lee, Taejeoung Kim, Jongwon Park, Jinpyo Kim, Pyongwon Suh, Ryan H. Choi, Sangwoo Lee, Jaejin Lee
arXiv Machine Learning
Sep 23

Accelerating the Mitigation of LLM Inference Nondeterminism Across GPU Architectures

The paper addresses the problem of non‑deterministic outputs from large language models (LLMs) when run on different GPU architectures, caused by floating‑point non‑associativity and hardware‑dependent kernel choices. It proposes a set of fixed‑configuration fused‑upcast GEMM kernels that load 16‑bit weights, upcast to FP32, and perform IEEE‑754 compliant reductions in a problem‑shape‑dependent order, ensuring identical linear‑layer outputs across NVIDIA Ampere, Ada, and Hopper GPUs. The new approach achieves 1.17–3.1× faster end‑to‑end performance than existing solutions and halves weight‑memory traffic while maintaining cross‑architecture reproducibility.

By Liam Cooper, Shinnung Jeong, Hyeran Jeon, Jeffrey Young, Hyesoon Kim
arXiv Machine Learning
Oct 28

DualPace: Decoupled Scheduling and Cooperative GPU--CPU Decoding for LLM Serving

DualPace is an LLM serving runtime that decouples scheduling between a GPU engine and an AMX‑enabled CPU engine, allowing the CPU to independently batch and advance decodes while still cooperating with the GPU for dense stages. This design expands key‑value cache capacity into host memory and reduces bursty CPU progress, leading to significant throughput gains—up to 43.2 % over NEO and 90.6 % over vLLM—without increasing latency. The system also maintains low time‑to‑first‑token and supports higher request rates at comparable tail latency.

By YuXuan Lin, KeQiang Wu, Kingsum Chow