arXiv:2607. 24377v1 Announce Type: cross Abstract: The quadratic cost of attention is a major bottleneck in diffusion-based video generation models.
Low-bit attention accelerates Transformer inference by moving the $QK^\top$ and $PV$ matrix multiplications to FP8 or FP4 matrix engines. However, the softmax path often evaluates shifted-score expone...
EFQ-Softmax is a low‑bit probability‑generation technique that replaces the traditional exp‑then‑quantize path in Transformer attention. It maps shifted attention scores directly to block‑scaled E2M1 operands using an exponent‑only scale and a single affine rule, allowing the same low‑bit representation to be used for both numerator and denominator updates. Experiments on Qwen3‑8B, Qwen3‑VL‑8B‑Instruct, and WAN2.2‑TI2V‑5B show that EFQ‑Softmax maintains or improves model quality while reducing vector‑stage latency by about 40% on the A5 vector unit.
The paper investigates low‑bit quantization for Multimodal Large Language Models (MLLMs), showing that MXFP8 retains near‑lossless performance while 4‑bit formats like MXFP4 and HiF4 cause significant degradation. It identifies activation quantization as the main source of this loss and introduces Residual Fallback Quantization (RFQ), a lightweight framework that adds a quantized residual pathway to improve activation fidelity without architectural changes. Experiments on Wan2.2 and Qwen3‑VL demonstrate that RFQ recovers much of the performance gap to BF16 baselines across generation and reasoning tasks.
arXiv:2606. 07618v1 Announce Type: cross Abstract: NVFP4 is a recently introduced hardware-supported FP4 format that improves the fidelity of 4-bit quantization through fine-grained block scales.
arXiv:2508.03351v3 Announce Type: replace-cross Abstract: Large language models (LLMs) have demonstrated remarkable capabilities across diverse language tasks, motivating their extension to vision-la...
arXiv:2609.15810v1 Announce Type: new Abstract: Diffusion Transformers deliver state-of-the-art video generation, but their long spatiotemporal sequences make attention the dominant deployment cost,...
arXiv:2609.26425v1 Announce Type: new Abstract: KV cache memory has become a major deployment bottleneck for video generation and world models, which motivates low-bit quantization study for efficien...
The paper introduces H-Scale, a lightweight post-processing technique for refining per-group scaling factors in NVFP4 quantized large language models. By using a diagonal second-order proxy from calibration activations, H-Scale selects hardware-valid scales that directly target layer output perturbation rather than just weight reconstruction error. Experiments on mainstream LLMs show that H-Scale improves NVFP4 baselines and brings several variants closer to BF16 performance without adding inference overhead.
arXiv:2607. 24953v1 Announce Type: cross Abstract: Reducing training precision is a key lever for improving the e ciency of large language model (LLM) training, but pushing beyond FP8 to 4-bit oating point (FP4) remains challenging due to instability during optimization.
arXiv:2609.37001v1 Announce Type: cross Abstract: Diffusion Transformers (DiTs) enable high-quality video generation but suffer from substantial inference latency, primarily attributable to the compu...
arXiv:2606. 04620v1 Announce Type: cross Abstract: LLMs have become the state-of-the-art algorithms for solving NLP tasks.
The paper introduces DSAQuant, a quantization‑aware training framework tailored for video diffusion models (VDMs). It aligns quantization with the denoising stages of VDMs, using denoising‑stage‑oriented supervision during training and denoising‑stage gated guidance during inference to preserve structure while improving detail reconstruction. Experiments on Wan and CogVideoX models under aggressive W3A3 and W4A4 quantization settings show that DSAQuant outperforms state‑of‑the‑art QAT baselines, boosting VBench scores by up to 6.60 while maintaining strong text‑video alignment.