The paper presents a memory‑efficient sparse‑binary self‑organising map (SOM) that scales a MEDLINE atlas to over a million neurons on a single consumer GPU. By re‑ordering the codebook into a feature‑major layout, the authors accelerate the best‑matching‑unit search by 4.5–8.5× without increasing quantisation error, enabling training of a 1,048,576‑neuron SOM in 72 s on a 24 GB GPU. The approach outperforms existing cuSPARSE and CPU‑based SOM implementations, achieving the largest SOM reported to date and demonstrating that resolution limits are computational rather than data‑driven.
By Andrew James Amos
arXiv:2604. 23466v2 Announce Type: replace Abstract: NVIDIA's CUDA Tile (CuTile) introduces a Python-based, tile-centric abstraction for GPU kernel development that aims to simplify programming while retaining Tensor Core and Tensor Memory Accelerator (TMA) efficiency on modern GPUs.
By Divakar Kumar Yadav, Tian Zhao, Deepak Kumar
arXiv:2607. 14568v1 Announce Type: cross Abstract: A companion study ran a 35B mixture-of-experts model on a 2011 NVIDIA Tesla C2075 (Fermi, sm_20, 6GB) as a GPU-prefill/CPU-decode hybrid, because the 4-bit model did not fit in device memory (arXiv:2606.
By A. C. Opus, J. Q. Lu
Serving an LLM efficiently depends on the engine, how the model is split across GPUs, precision, scheduler settings, speculative decoding, KV-cache management, and prefill-decode disaggregation. The b...
arXiv:2608.21157v1 Announce Type: cross
Abstract: High-performance GPU kernels underpin modern deep learning and scientific computing. As workloads become increasingly diverse and GPU hardware evolve...
By Jinghao Wang, Qiqi Gu, Chenpeng Wu, Jianguo Yao, Haibing Guan, Xijun Li
The paper introduces Primus Projection, a hybrid engine that combines GPU microbenchmarks with analytical scaling to predict multi‑node distributed training performance, and Primus Tuning Agent, an automated search that uses the projection engine as a fast oracle to explore the vast configuration space of parallelism and micro‑batching. Together, they reduce the need for costly trial‑and‑error on hundreds of GPUs, achieving predictions within ~10% of real throughput and enabling significant throughput gains—up to +27% in a Mixtral 8x22B case study—within minutes of single‑node exploration.
By Anshu Raina, Peyman Razaghi, Yuankai Chen, Cheng Yao, Yao Fu, Vidushi Goyal, Devang Patel, Wen Xie, Zhenyu Gu, Emad Barsoum
PageWeaver introduces a KV‑guided query union strategy for sparse attention, assembling query groups based on selected‑page affinity while preserving each query’s support and output ownership. The design employs a bounded GPU search to generate query IDs and an ID‑aware two‑CTA kernel that consumes them without materializing reordered Q tensors or cross‑page partial outputs. Experiments on the H200 Union8 with FP8 KV show a 1.70× speedup over FlashInfer, with additional latency reductions of 3.26–7.66% on selected captures and modest throughput gains for whole‑model prefill.
By Zhiyuan Li, Zihan Li, Zefang Yuan, Lei Wang, Hao Wang
Configuring distributed LLM training across hundreds of GPUs requires jointly optimizing tensor, pipeline, expert, context, and data parallelism alongside micro-batch size, recomputation, sharding, an...
The paper compares two strategies for handling memory limits in large language model (LLM) serving: tensor parallelism, which distributes weights and KV cache across multiple GPUs, and KV compression, which reduces cache size via quantisation and eviction on a single GPU. Using a cost‑normalised simulator calibrated on A100, A40, and H100 hardware, the authors find that across two models (Llama‑2 7B and 70B) and various GPU configurations, compression consistently outperforms tensor parallelism in cost per million tokens, offering 1.20× to 2.00× savings. The study identifies a model‑size threshold (~36B parameters on an 80 GB card) where compression dominates, while tensor parallelism becomes necessary only for larger models where weights alone exceed a single GPU’s capacity.
By Srikanta Datta Tumkur, Mehar Simhadri, Anshu Bansal, Jay Iyer, Sai Pavan Kumar, Sai Kapil Kumar, Ramesh Nampelly, Raj Dandekar
arXiv:2609.26147v1 Announce Type: new
Abstract: Modern LLMs are deployed as families of post-trained variants (base, instruct, chat, code) derived from a shared set of pre-trained weights. We present...
By Zhaohui Wang
arXiv:2603. 16428v2 Announce Type: replace-cross Abstract: Fine-tuning Large Language Models (LLMs) has become essential for domain adaptation, but its memory-intensive property exceeds the capabilities of most GPUs.
By Ruijia Yang, Zeyi Wen
ENAS is a hardware‑aware neural architecture search framework tailored for TinyML on microcontrollers. It uses a static feasibility check, a cell‑based search space with various block types and skip connections, and a three‑stage hybrid search strategy (random → top‑K → mutation) with cross‑run caching. The framework runs efficiently without GPUs, achieving significant search‑time speedups and competitive accuracy on Visual Wake Words and Melanoma Cancer benchmarks across a range of microcontrollers.
By Mohd Moin Khan, Naman Srivastava, Pandarasamy Arjunan