Hugging Face Trending Papers

Understanding and Exploiting Diagonal Attention Sparsity in Autoregressive Image Generation

arXiv Computer Vision
Sep 18

Understanding and Exploiting Diagonal Attention Sparsity in Autoregressive Image Generation

The paper investigates how attention sparsity behaves in autoregressive image generation, finding a distinct diagonal sparsity pattern due to spatial locality of visual tokens. It introduces a diagonal‑aware sparse attention mechanism that skips KV entries along the diagonal within a recent window, achieving up to 3.1× higher throughput and 1.19× lower latency with less than 2% quality loss compared to dense inference.

By Daeun Kim, Junwha Hong, Changhun Oh, Yoonsung Kim, Yoonhyeong Lee, Jongse Park
arXiv Machine Learning
Jul 2

Information-Regularized Attention for Visual-Centric Reasoning

arXiv:2607. 00434v1 Announce Type: cross Abstract: Vision-language models (VLMs) have become a paradigm for multimodal learning, yet remain unstable due to object hallucination, weak visual grounding, and catastrophic forgetting after full-parameter instruction tuning.

By Guohao Sun, Xiaofang Wang, Yash Patel, Mengchen Liu, Zhiqiang Tao, Praveen Krishnan
arXiv Computation and Language
Oct 9

GUI-KV: Efficient GUI Agents via KV Cache with Spatio-Temporal Awareness

GUI-KV is a plug‑and‑play key‑value cache compression method designed for graphical user interface agents that use vision‑language models. It exploits spatial saliency and temporal redundancy to prune cache entries, achieving near‑full‑cache accuracy while using modest memory budgets. Experiments on standard GUI agent benchmarks show significant reductions in decoding FLOPs and improvements in step accuracy compared to existing compression baselines.

By Kung-Hsiang Huang, Haoyi Qiu, Yutong Dai, Caiming Xiong, Chien-Sheng Wu