Hugging Face Trending Papers
EFQ-Softmax: Exp-Free Quantization for Softmax
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv Machine Learning
Sep 21
EFQ-Softmax: Exp-Free Quantization for Softmax
EFQ-Softmax is a low‑bit probability‑generation technique that replaces the traditional exp‑then‑quantize path in Transformer attention. It maps shifted attention scores directly to block‑scaled E2M1 operands using an exponent‑only scale and a single affine rule, allowing the same low‑bit representation to be used for both numerator and denominator updates. Experiments on Qwen3‑8B, Qwen3‑VL‑8B‑Instruct, and WAN2.2‑TI2V‑5B show that EFQ‑Softmax maintains or improves model quality while reducing vector‑stage latency by about 40% on the A5 vector unit.
arXiv Computer Vision
Sep 15
VC-Attention: Value Smoothing and Softmax Casting for Low-bit Attention
arXiv:2609.15810v1 Announce Type: new Abstract: Diffusion Transformers deliver state-of-the-art video generation, but their long spatiotemporal sequences make attention the dominant deployment cost,...
arXiv AI
Jul 7
HiFA4: Training-Free 4-bit FlashAttention on Ascend HIF4 NPUs for LLM Inference
arXiv:2607. 04302v1 Announce Type: cross Abstract: We present HiFA4, a post-training operator-level design that executes both QK^T and PV in FlashAttention as 4-bit HIF4 Cube GEMMs for LLM inference on Ascend NPUs, while maintaining the online softmax state in FP16.
arXiv AI
Jul 28
MXAttention: Data-Free Optimal Scaling and Pre-Normalization Quantization for MXFP4 Attention
arXiv:2607. 24377v1 Announce Type: cross Abstract: The quadratic cost of attention is a major bottleneck in diffusion-based video generation models.
arXiv AI
Sep 4
Why Gated DeltaNet Survives 4-Bit Quantization: NVFP4 W4A4 for the Recurrent Half of a Hybrid 27B LLM
The paper presents a 4‑bit quantization recipe, Minima: NVFP4 W4A4, that fully quantizes all linear layers—including the Gated DeltaNet (GDN) recurrent blocks—of the 27‑billion‑parameter Qwen3.8 LLM. Across a suite of benchmarks (perplexity, MMLU‑Pro, GSM8K, AIME'25, GPQA‑Diamond, LiveCodeBench, and RULER retrieval), the quantized model matches BF16 performance within seed noise while being 17.5 GiB in size and 14–19 % faster at prefill. The authors attribute this success to four mechanisms: block‑scaling of residuals, robust gate projections, the delta‑rule recurrence’s noise‑plateau behavior, and the per‑token quantization cost’s dilution over long contexts.
arXiv AI
Jun 4
QuBLAST: A Framework for Quantizing Large Language Models with Block-Level Compression Approach and Activation Scaling Strategy
arXiv:2606. 04620v1 Announce Type: cross Abstract: LLMs have become the state-of-the-art algorithms for solving NLP tasks.