arXiv Computer Vision
VC-Attention: Value Smoothing and Softmax Casting for Low-bit Attention
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
Hugging Face Trending Papers
Sep 9
EFQ-Softmax: Exp-Free Quantization for Softmax
Low-bit attention accelerates Transformer inference by moving the $QK^\top$ and $PV$ matrix multiplications to FP8 or FP4 matrix engines. However, the softmax path often evaluates shifted-score expone...
arXiv AI
Jun 4
Stochastic Sparse Attention for Memory-Bound Inference
arXiv:2605. 01910v2 Announce Type: replace-cross Abstract: Autoregressive decoding becomes bandwidth-limited at long contexts, as generating each token requires reading all $n_k$ key and value vectors from KV cache.
arXiv AI
Jul 7
HyperVAttention: Efficient Sparse Attention with Spatio-Temporal Clustering for Video Diffusion
arXiv:2607. 03012v1 Announce Type: cross Abstract: Video Diffusion Transformers (VDiTs) have demonstrated significant capabilities in high-fidelity video generation.
arXiv AI
Aug 13
LoSA: Near-Lossless Sparse Attention for Training-Free Video Diffusion Acceleration
arXiv:2608. 12032v1 Announce Type: cross Abstract: Video diffusion transformers are costly to sample: every denoising step applies self-attention over a long 3D token sequence, a quadratic cost that dominates as resolution and duration grow.
arXiv AI
Aug 10
Fast LapSum: Exact Differentiable Top-k at Million Scale
arXiv:2608. 06912v1 Announce Type: new Abstract: The top-$k$ operation is a fundamental building block of modern sparse computation, enabling token routing, expert activation, memory selection, and attention pruning.
arXiv AI
Jun 15
Fast Autoregressive Video Diffusion and World Models with Temporal Cache Compression and Sparse Attention
arXiv:2602. 01801v2 Announce Type: replace-cross Abstract: Autoregressive video diffusion models enable streaming generation, opening the door to long-form synthesis, video world models, and interactive neural game engines.