The paper investigates whether discrete visual tokens can preserve semantic information as effectively as continuous representations. Contrary to the common belief that quantization error inevitably causes semantic loss, the authors show that larger quantization errors can still yield better multimodal understanding, and that the effective bit budget of discrete indices is the critical factor. They propose Factorized Vector Quantization, which uses parallel codebooks to encode complementary views of visual features, achieving performance that matches continuous representations and surpasses prior discrete models on a broad set of metrics.
By Jialv Zou, Xiaotong Li, Zhengrong Yue, Xu Huang, Yaxin Luo, Wenhao Yan, Haodong Li, Wenyu Liu, Xinggang Wang
arXiv:2609.39451v1 Announce Type: new
Abstract: Progressive autoregressive image codecs provide an appealing paradigm for generative compression by quantizing continuous latents into discrete tokens,...
By Qin Yan, Ruixiao Dong, Yutao Xie, Li Li, Ying Chen, Kai Li, Daowen Li, Houqiang Li
arXiv:2512. 08240v2 Announce Type: replace-cross Abstract: Vision-language models (VLMs) rely on hundreds of visual tokens, leading to high computational and memory costs.
By Jusheng Zhang, Xiaoyang Guo, Tongyu Mo, Qinhan Lv, Wenhao Chai, Jian Wang, Keze Wang, Liang Lin
VQ-Transplant is a framework that allows new vector‑quantization (VQ) modules to be inserted into frozen, pre‑trained visual tokenizers without retraining the entire model. By preserving all encoder‑decoder parameters and adding a lightweight decoder adaptation trained for only five epochs on ImageNet‑1k, the method mitigates decoder‑quantization mismatch. Experiments show that VQ-Transplant achieves near state‑of‑the‑art reconstruction fidelity for industry‑level models such as VAR while cutting training costs by 95%.
By Xianghong Fang, Yuan Yuan, Dehan Kong, Tim G. J. Rudner
VTBench is a new benchmark that evaluates visual tokenizers (VTs) used in autoregressive image generation. It assesses VTs on image reconstruction, detail preservation, and text preservation across diverse scenarios, revealing that continuous VAEs outperform discrete VTs in maintaining spatial structure and semantic detail. The study also explores GPT‑4o’s potential autoregressive behavior and releases the benchmark publicly to encourage development of robust, open‑source VTs.
By Huawei Lin, Tony Geng, Zhaozhuo Xu, Weijie Zhao
arXiv:2605. 26089v2 Announce Type: replace-cross Abstract: We present Channel-wise Vector Quantization (CVQ), a novel image tokenization paradigm that replaces patch-wise tokens with channel-wise tokens.
By Wei Song, Tianhang Wang, Yitong Chen, Tong Zhang, Zuxuan Wu, Min Li, Jiaqi Wang, Kaicheng Yu