The paper investigates whether discrete visual tokens can preserve semantic information as effectively as continuous representations. Contrary to the common belief that quantization error inevitably causes semantic loss, the authors show that larger quantization errors can still yield better multimodal understanding, and that the effective bit budget of discrete indices is the critical factor. They propose Factorized Vector Quantization, which uses parallel codebooks to encode complementary views of visual features, achieving performance that matches continuous representations and surpasses prior discrete models on a broad set of metrics.
By Jialv Zou, Xiaotong Li, Zhengrong Yue, Xu Huang, Yaxin Luo, Wenhao Yan, Haodong Li, Wenyu Liu, Xinggang Wang
arXiv:2609.39451v1 Announce Type: new
Abstract: Progressive autoregressive image codecs provide an appealing paradigm for generative compression by quantizing continuous latents into discrete tokens,...
By Qin Yan, Ruixiao Dong, Yutao Xie, Li Li, Ying Chen, Kai Li, Daowen Li, Houqiang Li
arXiv:2512. 08240v2 Announce Type: replace-cross Abstract: Vision-language models (VLMs) rely on hundreds of visual tokens, leading to high computational and memory costs.
By Jusheng Zhang, Xiaoyang Guo, Tongyu Mo, Qinhan Lv, Wenhao Chai, Jian Wang, Keze Wang, Liang Lin
VQ-Transplant is a framework that allows new vector‑quantization (VQ) modules to be inserted into frozen, pre‑trained visual tokenizers without retraining the entire model. By preserving all encoder‑decoder parameters and adding a lightweight decoder adaptation trained for only five epochs on ImageNet‑1k, the method mitigates decoder‑quantization mismatch. Experiments show that VQ-Transplant achieves near state‑of‑the‑art reconstruction fidelity for industry‑level models such as VAR while cutting training costs by 95%.
By Xianghong Fang, Yuan Yuan, Dehan Kong, Tim G. J. Rudner
VTBench is a new benchmark that evaluates visual tokenizers (VTs) used in autoregressive image generation. It assesses VTs on image reconstruction, detail preservation, and text preservation across diverse scenarios, revealing that continuous VAEs outperform discrete VTs in maintaining spatial structure and semantic detail. The study also explores GPT‑4o’s potential autoregressive behavior and releases the benchmark publicly to encourage development of robust, open‑source VTs.
By Huawei Lin, Tony Geng, Zhaozhuo Xu, Weijie Zhao
arXiv:2605. 26089v2 Announce Type: replace-cross Abstract: We present Channel-wise Vector Quantization (CVQ), a novel image tokenization paradigm that replaces patch-wise tokens with channel-wise tokens.
By Wei Song, Tianhang Wang, Yitong Chen, Tong Zhang, Zuxuan Wu, Min Li, Jiaqi Wang, Kaicheng Yu
Tree-VQ introduces a progressive tree‑structured vector quantization framework for learned image compression, organizing discrete codewords in a hierarchical binary tree where each latent token is represented by a routed root‑to‑leaf path. Every prefix of this path yields a valid quantized representation, enabling coarse reconstructions from shallow nodes and successive refinements from deeper nodes. The method incorporates a prefix‑compatible tree entropy model, rate‑aware refinement scheduling, and hierarchical prefix supervision to achieve efficient, low‑latency compression with superior perceptual quality and fewer parameters compared to existing approaches.
By Xinkun Wang, Tianyi Xu, Qingyu Luo, Mingming Ma, Changzhe Jiao, Fu Li, Yi Niu
arXiv:2411.16119v2 Announce Type: replace-cross
Abstract: It is customary to deploy uniform scalar quantization in the end-to-end optimized Neural image compression methods, instead of more powerful...
By Xi Zhang, Xiaolin Wu
arXiv:2611.05380v1 Announce Type: new
Abstract: Visual autoregressive generation spans two dominant paradigms: next-scale prediction for images and video, and next-frame prediction for video. Both de...
By Boxun Xu, Jiaji Lu, Yuxuan Yin, Zihu Wang, Ziyue Liu, Yu Wang, Zirui Liu, Peng Li
arXiv:2605.19660v3 Announce Type: replace
Abstract: The rapid advancement toward long-context reasoning and multi-modal intelligence has made the memory footprint of the Key-Value (KV) cache a domina...
By Zunhai Su, Rui Yang, Chao Zhang, Yaxiu Liu, Yifan Zhang, Wei Wu, Jing Xiong, Dayou Du, Xialie Zhuang, Yulei Qian, Jianchao Tan, Yuchen Xie, Yik-Chung Wu, Hongxia Yang, Ngai Wong
arXiv:2608. 04074v1 Announce Type: cross Abstract: Long-context LLM decoding reads the key-value (KV) cache at every step.
By Samuel Fern\'andez-Mendui\~na, Amir Ziashahabi, Eduardo Pavez, Antonio Ortega, Salman Avestimehr
StableVQ introduces practical guidelines to improve training stability for vector‑quantized tokenizers used in image generation models. It addresses instability caused by the entanglement of encoder–decoder and codebook training by proposing three techniques: Dynamic STE for the encoder, Region VQ Loss for the codebook, and a Decoupled Schedule for independent learning rates. Experiments on ImageNet show consistent gains in stability, codebook utilization, and reconstruction quality across various settings.
By Bao Tang, Jiahao Guo, Haoxiang Cao, Wenyu Liu, Changqian Yu, Kun Gai, Xinggang Wang