A Gentle Introduction to 8-bit Matrix Multiplication for transformers at scale using transformers, accelerate and bitsandbytes

Overview of natively supported quantization schemes in 🤗 Transformers

Introducing Optimum: The Optimization Toolkit for Transformers at Scale

Memory-efficient Diffusion Transformers with Quanto and Diffusers

On the Expressive Power of Transformers

arXiv:2608. 12671v1 Announce Type: new Abstract: Multi-layer transformers form the critical component of essentially all large language models (LLMs) in use today.

Modern analog computing for solving differential and matrix equations

arXiv:2606. 13179v1 Announce Type: cross Abstract: In recent years, driven by the computational demands of data-intensive applications such as artificial intelligence and scientific computing, analog computing has gained renewed interest.

Native-speed vLLM transformers modeling backend

Modern analog computing for solving differential and matrix equations

In recent years, driven by the computational demands of data-intensive applications such as artificial intelligence and scientific computing, analog computing has gained renewed interest. Given the diversity of computational tasks and recent advancements in analog CMOS circuits and resistive memory technologies, we refer to the evolving landscape as modern analog computing.

KroQuant: Kronecker-Structured Block Transforms for Efficient Post-Training Quantization of Diffusion Transformers

arXiv:2607. 21446v1 Announce Type: new Abstract: Post-training quantization (PTQ) of diffusion transformers (DiTs) to W4A4 severely degrades output quality, because activations entering each linear layer contain outliers that 4-bit formats cannot represent.

Learning Fine-grained Parameter Sharing via Sparse Tensor Decomposition

arXiv:2411. 09816v5 Announce Type: replace Abstract: Large neural networks achieve state-of-the-art performance on many tasks, yet their sheer size hinders deployment on resource-constrained devices.

Hugging Face and Graphcore partner for IPU-optimized Transformers

The Discrete-Log Clock: How a Transformer Learns Modular Multiplication

arXiv:2606. 17399v1 Announce Type: cross Abstract: When small transformers grok modular multiplication, prior work reports that the learned embedding has a "dense" Fourier spectrum requiring all frequencies.

NIFA: Nonlinear IMC enhanced FPGA for efficient ML inference

arXiv:2607. 15123v1 Announce Type: cross Abstract: Recent FPGAs have improved deep learning (DL) inference efficiency through dedicated tensor blocks and in-BRAM computation.