arXiv AI
T-CCL: Resource Efficient and Performant Collective Communication using Tensor Memory Accelerator
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Sep 15
mKernel: Fast Multi-GPU, Multi-Node Fused Kernels
arXiv:2609.13585v1 Announce Type: cross Abstract: Communication has become a bottleneck in distributed training and inference of large models. Overlapping communication with computation at the granul...
arXiv Machine Learning
Jul 10
CTA-Pipelining: A Latency-Oriented Spatial Scaling Method for Multi-GPU Systems
arXiv:2607. 07862v1 Announce Type: cross Abstract: The evolution of compute infrastructure has transformed multi-GPU systems into tightly integrated shared-memory structures.
arXiv Machine Learning
6d ago
Efficient Expert-Parallel Communication on PCIe-Connected Consumer GPUs
ThunderEP is a new communication design for expert parallelism on PCIe-connected consumer GPUs that eliminates relay hops, uses DMA engines to avoid GPU compute contention, and reduces CPU polling overhead. Integrated into vLLM, it outperforms NCCL on RTX 4090 and RTX 5090 GPUs, delivering average speedups of 2.00× for dispatch, 1.53× for combine, and up to 1.66× end‑to‑end over existing MoE inference frameworks.
arXiv Machine Learning
Jun 5
Evaluating CUDA Tile for AI Workloads on Hopper and Blackwell GPUs
arXiv:2604. 23466v2 Announce Type: replace Abstract: NVIDIA's CUDA Tile (CuTile) introduces a Python-based, tile-centric abstraction for GPU kernel development that aims to simplify programming while retaining Tensor Core and Tensor Memory Accelerator (TMA) efficiency on modern GPUs.
arXiv AI
Sep 30
Purlin: Separating Orchestration from the Datapath of Collectives
arXiv:2609.36954v1 Announce Type: cross Abstract: Distributed inference depends on GPU collective communication that must keep pace with evolving hardware and specialized workloads. However, existing...
arXiv Machine Learning
Jun 2
DuetServe: Harmonizing Prefill and Decode for LLM Serving via Adaptive GPU Multiplexing
arXiv:2511. 04791v2 Announce Type: replace Abstract: Modern LLM serving systems must sustain high throughput while meeting strict latency SLOs across two distinct inference phases: compute-intensive prefill and memory-bound decode phases.