Accelerate ND-Parallel: A guide to Efficient Multi-GPU Training

SUPREME: A Multi-GPU Framework for Reproducible Image Unlearning Method Evaluation

arXiv:2606. 00380v1 Announce Type: cross Abstract: Machine unlearning removes the influence of specific training data from a trained model without retraining it from scratch.

SparseDitto: Customizing GPU Kernels for Different Sparsity Patterns with LLM-Based Agentic System

arXiv:2608. 05033v1 Announce Type: cross Abstract: Sparse matrix kernels are fundamental to scientific computing, graph analytics, and machine learning.

GPU Parallelization Strategies for Forward and Backward Propagation in Shallow Neural Networks: A CUDA-Based Comparative Study

arXiv:2606. 30497v1 Announce Type: cross Abstract: We present a comparative study of CUDA optimization strategies applied to forward and backward propagation in a shallow neural network.

TorchKM: A GPU-Oriented Library for Kernel Learning and Model Selection

arXiv:2606. 06742v1 Announce Type: new Abstract: TorchKM is an open-source library for kernel machines, including support vector machines, kernel logistic regression, and kernel quantile regression, with GPU acceleration.

Mixture-of-Parallelisms: Towards Memory-Efficient Training Stack for Mixture-of-Experts Models

arXiv:2607. 01844v1 Announce Type: cross Abstract: This paper showcases a memory-efficient training stack for Mixture-of-Experts (MoE) models.

An Efficient Heterogeneous Co-Design for Fine-Tuning on a Single GPU

arXiv:2603. 16428v2 Announce Type: replace-cross Abstract: Fine-tuning Large Language Models (LLMs) has become essential for domain adaptation, but its memory-intensive property exceeds the capabilities of most GPUs.

SparseDitto: An Agentic Sparse Compilation Framework through Architecture-Aware Synthesis on GPUs

SparseDitto is an agentic sparse compilation framework that jointly synthesizes representation, execution schedule, and hardware mapping for sparse matrix computations on GPUs. It uses structural analysis, a learned template-ranking prior, and LLM-guided lowering to generate CUDA code, with target-GPU profiling refining the plan. The framework supports multiple operators such as SpMV, SpMM, and SpGEMM, adapts to different hardware, and achieves significant speedups over cuSPARSE, including up to 146.61× on certain matrices and 3.39× acceleration for full-batch GCN training.

CTA-Pipelining: A Latency-Oriented Spatial Scaling Method for Multi-GPU Systems

arXiv:2607. 07862v1 Announce Type: cross Abstract: The evolution of compute infrastructure has transformed multi-GPU systems into tightly integrated shared-memory structures.

HIERA: Workload-Aware Planning Across Implementation Spaces for GPU Kernel Optimization

arXiv:2608.21157v1 Announce Type: cross Abstract: High-performance GPU kernels underpin modern deep learning and scientific computing. As workloads become increasingly diverse and GPU hardware evolve...

From Zero to GPU: A Guide to Building and Scaling Production-Ready CUDA Kernels

Resource-aware Computation-Communication Overlap for multi-GPU ML Workloads

arXiv:2606. 09200v1 Announce Type: cross Abstract: The rapid growth of large-scale machine learning (ML) has made distributed training across multiple GPUs a fundamental component of modern ML systems.

Hugging Face and AMD partner on accelerating state-of-the-art models for CPU and GPU platforms