arXiv Machine Learning By Andrew James Amos

From 80x to 385x: A Best-Matching-Unit Search at the L2 Roof, Measured Against a Symmetrically Tuned Baseline

Read the original on arXiv Machine Learning →

The paper reports a comprehensive tuning of both a novel sparse self‑organizing map algorithm (SparseBin) and its baseline cuSPARSE implementation. By optimizing four key levers—tile size, tile‑membership clustering, neuron‑axis chunking, and vectorised loads—the authors achieved a 5.6‑10.1× speed‑up per epoch for map sizes ranging from 32×32 to 512×512, and increased the performance margin over the CUDA baseline from ~80× to ~385×. The tuned kernel saturated the L2 bandwidth at 77% of peak, indicating that further performance gains are unlikely without new hardware or fundamentally different approaches.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning
Aug 26

A Feature-Major Codebook for Memory-Efficient Sparse-Binary Self-Organizing Maps: Scaling a MEDLINE Atlas to 1.05 Million Neurons on a Single Consumer GPU

The paper presents a memory‑efficient sparse‑binary self‑organising map (SOM) that scales a MEDLINE atlas to over a million neurons on a single consumer GPU. By re‑ordering the codebook into a feature‑major layout, the authors accelerate the best‑matching‑unit search by 4.5–8.5× without increasing quantisation error, enabling training of a 1,048,576‑neuron SOM in 72 s on a 24 GB GPU. The approach outperforms existing cuSPARSE and CPU‑based SOM implementations, achieving the largest SOM reported to date and demonstrating that resolution limits are computational rather than data‑driven.

By Andrew James Amos
arXiv AI
3d ago

Beyond the Trial-and-Error Loop: Hybrid Projection and Automated Tuning for Distributed Training

The paper introduces Primus Projection, a hybrid engine that combines GPU microbenchmarks with analytical scaling to predict multi‑node distributed training performance, and Primus Tuning Agent, an automated search that uses the projection engine as a fast oracle to explore the vast configuration space of parallelism and micro‑batching. Together, they reduce the need for costly trial‑and‑error on hundreds of GPUs, achieving predictions within ~10% of real throughput and enabling significant throughput gains—up to +27% in a Mixtral 8x22B case study—within minutes of single‑node exploration.

By Anshu Raina, Peyman Razaghi, Yuankai Chen, Cheng Yao, Yao Fu, Vidushi Goyal, Devang Patel, Wen Xie, Zhenyu Gu, Emad Barsoum