arXiv Machine Learning
Jun 16
Shift-and-Sum Quantization for Visual Autoregressive Models
arXiv:2606. 16131v1 Announce Type: cross Abstract: Post-training quantization (PTQ) enables efficient deployment of deep networks using a small set of data.
Channel-wise Vector Quantization
arXiv:2605. 26089v2 Announce Type: replace-cross Abstract: We present Channel-wise Vector Quantization (CVQ), a novel image tokenization paradigm that replaces patch-wise tokens with channel-wise tokens.
Related stories
arXiv AI
Aug 11
UniSpace: Unified Visual Representation and Scalable Multimodal Modeling
arXiv:2608. 08676v1 Announce Type: cross Abstract: Semantic vision encoders have become a central visual interface for multimodal understanding and semantic conditioning in image generation.
arXiv Computer Vision
Sep 16
Structure-Detail Decoupled Autoregressive Generation for Fast and High-Fidelity Virtual Try-On
arXiv:2607.11233v2 Announce Type: replace Abstract: Virtual try-on (VTON) is a bi-conditional image generation problem that requires not only accurate person preservation but also faithful garment de...
Hugging Face Trending Papers
Jun 9
IDEAL: In-DEpth ALignment Makes A Discrete Representation AutoEncoder
Built on pretrained vision foundation models (VFMs), representation autoencoders (RAEs) have recently emerged as a promising approach for constructing semantically rich latent spaces for image generation. However, their reconstruction quality often remains suboptimal, largely because deep VFM representations do not preserve sufficient fine-grained visual detail.
arXiv AI
Jun 26
Safe Autoregressive Image Generation with Iterative Self-Improving Codebooks
arXiv:2606. 27147v1 Announce Type: cross Abstract: Unlike diffusion-based models that operate in continuous latent spaces, autoregressive unified multimodal models produce images by sequentially predicting discretized visual tokens.
Hugging Face Trending Papers
Jul 27
UniGen-AR: Unifying Visual Generation with Auto-Regressive Modeling
Modern computer vision pipelines remain fragmented, with tasks such as text-to-image generation, editing, restoration, and classical perception handled by separate models. We study Unified Visual Generation (UVG), where a single model produces diverse image-valued outputs through a unified multimodal interface.
arXiv Machine Learning
Sep 3
A Unified Rate-Distortion Perspective on Vector, Product, and Scalar Quantization
The paper introduces a unified rate–distortion framework for discrete visual tokenization, encompassing vector, product, and scalar quantization. It shows that minimizing distortion, rather than maximizing codebook utilization, is the key objective for reconstruction fidelity and establishes fairness conditions for comparing quantizers. Under these conditions, the study confirms the distortion hierarchy VQ–PQ–SQ and demonstrates that modern VQ methods achieve the lowest distortion.
arXiv AI
2d ago
LensVLM: Selective Context Expansion for Compressed Visual Representation of Text
LensVLM is an inference framework and post‑training recipe that lets Vision‑Language Models (VLMs) process compressed images of text by selectively expanding only the relevant parts back to full resolution. Using Qwen3.5‑9B‑Base, LensVLM achieves accuracy comparable to full‑text models at 4.3× compression and outperforms other compression baselines up to 10.1× across seven text QA benchmarks, while also improving performance on multimodal document and code tasks as compression increases.
arXiv Computer Vision
Aug 25
VQ-Transplant: Efficient VQ-Module Integration for Pre-trained Visual Tokenizers
VQ-Transplant is a framework that allows new vector‑quantization (VQ) modules to be inserted into frozen, pre‑trained visual tokenizers without retraining the entire model. By preserving all encoder‑decoder parameters and adding a lightweight decoder adaptation trained for only five epochs on ImageNet‑1k, the method mitigates decoder‑quantization mismatch. Experiments show that VQ-Transplant achieves near state‑of‑the‑art reconstruction fidelity for industry‑level models such as VAR while cutting training costs by 95%.
arXiv Computer Vision
Sep 24
UVU: Improving Multimodal Understanding via Vision-Language Unified Autoregressive Paradigm
UVU is a vision-language unified autoregressive framework that integrates visual supervision directly into the pre-training stage of multimodal large language models. By using continuous visual encoding and a large-scale iterative hierarchical clustering algorithm to build a pixel-level visual codebook, UVU enables lossless representation of visual inputs and autoregressive generation of pixel-level image tokens alongside textual tokens. This approach synergizes pixel-level visual perception with semantic-level visual understanding, allowing models to internalize visual reconstruction capabilities and improve multimodal understanding performance.
arXiv AI
Jul 8
Rethinking Visual Autoregressive Sampling with Information-Grounding Guidance
arXiv:2509. 23876v3 Announce Type: replace-cross Abstract: Autoregressive (AR) models based on next-scale prediction have emerged as a powerful tool for image generation, but they face a critical weakness: information inconsistencies between patches across timesteps introduced by progressive resolution scaling.
arXiv AI
Aug 12
Hybrid Token Compression for Vision-Language Models
arXiv:2512. 08240v2 Announce Type: replace-cross Abstract: Vision-language models (VLMs) rely on hundreds of visual tokens, leading to high computational and memory costs.