KOSMOS is a library that enables self‑scheduling persistent kernels for transformer training, allowing workgroups to switch between communication and computation roles as data becomes ready without a scheduler or tuned split. It fuses all tensor and sequence parallelism collectives with GEMM operations and runs entire expert layers in a single launch per direction. On AMD Instinct GPUs, KOSMOS improves TP+SP end‑to‑end training in Megatron‑LM by up to 1.28× and accelerates DeepSeek‑V3 and Qwen3‑235B proxies by 1.29–1.83× over DeepEP.
arXiv:2607. 23264v1 Announce Type: cross Abstract: Fine-grained, device-initiated communication lets persistent GPU kernels in distributed diffusion transformer (DiT) inference issue remote stores and overlap data movement with Tensor Core computation.
arXiv:2609.13585v1 Announce Type: cross Abstract: Communication has become a bottleneck in distributed training and inference of large models. Overlapping communication with computation at the granul...
arXiv:2607. 19539v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) architectures increase model capacity without proportionally increasing computation cost and have become a key building block for scaling large language models (LLMs) to trillion-parameter regimes.
arXiv:2512. 22219v2 Announce Type: replace-cross Abstract: We introduce Mirage Persistent Kernel (MPK), the first compiler and runtime system that automatically transforms multi-GPU model inference into a single high-performance mega-kernel.
arXiv:2606. 01839v1 Announce Type: cross Abstract: LLM-based agents resolve a user task through many turns of dependent inference and tool calls, producing a workload whose total cost is unknown when the task arrives.
WiSP (Working‑Set Paging) is a routing‑aware expert pager that allows Mixture‑of‑Experts models to run on GPUs that cannot hold the entire expert pool by paging experts in and out of VRAM while preserving byte‑identical outputs. On a 24 GiB RTX 3090, WiSP doubles decode throughput compared to static offload when the model does not fit, and its companion policy MV‑WSA allocates VRAM between resident experts and KV cache based on marginal latency benefit, reducing end‑to‑end time by up to 1.19× without altering model outputs.
FastKernels is a benchmark comprising 384 GPU kernel tasks from 47 architectures across 8 categories, designed to evaluate kernel generation agents in realistic production settings. It covers 94.6% of HuggingFace Transformers architectures, scoring kernels on correctness, coverage, and speedup within their native execution paths. The benchmark reveals that kernel‑level speedups often collapse to modest end‑to‑end gains and that isolated performance rankings can misrepresent real‑world effectiveness.
Mixture-of-Experts (MoE) layers replace the feed-forward block of a Transformer with E expert networks, and each token is routed to k of these experts. Under expert parallelism (EP) the experts are di...
arXiv:2606. 24722v1 Announce Type: new Abstract: Frontier AI training is increasingly shaped by access to dense, centrally controlled accelerator clusters.
arXiv:2610.09372v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) layers replace the feed-forward block of a Transformer with E expert networks, and each token is routed to k of these experts....
We address a fundamental question: How far should kernel fusion extend and which boundaries are worth preserving? Unlike the general convention of using one monolith megakernel at lower batch sizes or...
arXiv:2511.11628v2 Announce Type: replace-cross Abstract: Modern operating system process schedulers typically enforce a static, ``one-policy-fits-all'' design, which inevitably compromises performan...