GaLore: Advancing Large Model Training on Consumer-grade Hardware

Accelerate Large Model Training using DeepSpeed

Intel and Hugging Face Partner to Democratize Machine Learning Hardware Acceleration

AMD + 🤗: Large Language Models Out-of-the-Box Acceleration with AMD GPU

MLSYSIM: First-Principles Infrastructure Modeling for Machine Learning Systems

arXiv:2607. 02558v1 Announce Type: cross Abstract: As machine learning shifts from laboratory curiosity to critical infrastructure, the systems that sustain it span an extraordinary range, from sub-milliwatt microcontrollers to multi-gigawatt datacenter fleets.

BluTrain: A C++/CUDA Framework for AI Systems

arXiv:2606. 24780v1 Announce Type: new Abstract: Progress in deep learning is, at scale, more a matter of systems engineering than of modelling: the behaviour of a model in training (its throughput, its memory footprint, and the numerical fidelity of the result) is determined less by the architecture itself than by how that architecture is expressed on the hardware.

Mixture-of-Parallelisms: Towards Memory-Efficient Training Stack for Mixture-of-Experts Models

arXiv:2607. 01844v1 Announce Type: cross Abstract: This paper showcases a memory-efficient training stack for Mixture-of-Experts (MoE) models.