arXiv:2606. 16131v1 Announce Type: cross Abstract: Post-training quantization (PTQ) enables efficient deployment of deep networks using a small set of data.
arXiv:2508.03351v3 Announce Type: replace-cross Abstract: Large language models (LLMs) have demonstrated remarkable capabilities across diverse language tasks, motivating their extension to vision-la...
arXiv:2605. 26089v2 Announce Type: replace-cross Abstract: We present Channel-wise Vector Quantization (CVQ), a novel image tokenization paradigm that replaces patch-wise tokens with channel-wise tokens.
VQ-Transplant is a framework that allows new vector‑quantization (VQ) modules to be inserted into frozen, pre‑trained visual tokenizers without retraining the entire model. By preserving all encoder‑decoder parameters and adding a lightweight decoder adaptation trained for only five epochs on ImageNet‑1k, the method mitigates decoder‑quantization mismatch. Experiments show that VQ-Transplant achieves near state‑of‑the‑art reconstruction fidelity for industry‑level models such as VAR while cutting training costs by 95%.
arXiv:2609.26425v1 Announce Type: new Abstract: KV cache memory has become a major deployment bottleneck for video generation and world models, which motivates low-bit quantization study for efficien...
UVU is a vision-language unified autoregressive framework that integrates visual supervision directly into the pre-training stage of multimodal large language models. By using continuous visual encoding and a large-scale iterative hierarchical clustering algorithm to build a pixel-level visual codebook, UVU enables lossless representation of visual inputs and autoregressive generation of pixel-level image tokens alongside textual tokens. This approach synergizes pixel-level visual perception with semantic-level visual understanding, allowing models to internalize visual reconstruction capabilities and improve multimodal understanding performance.
PACE introduces a training‑free Condense‑and‑Extract framework that speeds up Vision‑Language Model inference by first adaptively downsampling visual inputs before encoding and then selectively retaining essential tokens during decoding. The Adaptive Pixel Compressor (APC) reduces encoder workload while preserving global context, and the Dynamic Dual‑Attention Extractor (DDAE) keeps task‑critical details by fusing visual and language signals. Applied to Qwen2.5‑VL‑7B, PACE maintains 93.8% of performance using only 10% of visual tokens, achieving a 3.1× speedup in time to first token.
arXiv:2509. 23876v3 Announce Type: replace-cross Abstract: Autoregressive (AR) models based on next-scale prediction have emerged as a powerful tool for image generation, but they face a critical weakness: information inconsistencies between patches across timesteps introduced by progressive resolution scaling.
arXiv:2608. 13932v1 Announce Type: new Abstract: Iterative Generative Models (IGMs) span autoregressive and diffusion paradigms, and hybrid variants that couple them can achieve remarkable image-generation fidelity.
arXiv:2606. 04620v1 Announce Type: cross Abstract: LLMs have become the state-of-the-art algorithms for solving NLP tasks.
The paper investigates how attention sparsity behaves in autoregressive image generation, finding a distinct diagonal sparsity pattern due to spatial locality of visual tokens. It introduces a diagonal‑aware sparse attention mechanism that skips KV entries along the diagonal within a recent window, achieving up to 3.1× higher throughput and 1.19× lower latency with less than 2% quality loss compared to dense inference.
arXiv:2609.38166v1 Announce Type: cross Abstract: Recent LLMs increasingly adopt hybrid designs that replace standard attention with linear attention, such as Gated DeltaNet (GDN) and Kimi Delta Atte...
arXiv:2607. 00371v1 Announce Type: cross Abstract: Visual AutoRegressive modeling (VAR) has pioneered a coarse-to-fine multi-scale autoregressive generative paradigm, demonstrating strong capabilities in image generation.