MoETok introduces a sparse expert routing mechanism for 1D visual tokenizers, enabling distinct Content and Detail queries to specialize in high‑level semantics and fine‑grained visual details, respectively. By jointly optimizing reconstruction and semantic objectives, the router learns role‑specialized processing paths that improve reconstruction, semantic representation, and autoregressive generation across various model scales and token budgets. On ImageNet, MoETok achieves state‑of‑the‑art class‑conditional generation within the 1D tokenizer family and, when scaled to a unified autoregressive model, matches or surpasses diffusion‑based models on benchmarks such as DPG‑Bench, GenEval2, and WISE.
arXiv:2607. 25527v1 Announce Type: cross Abstract: Unifying visual understanding and generation in one model holds immense promise, but remains challenging and expensive due to heavy compute and data demands and conflicts between the visual features needed for these two capabilities.
VIVAS is a new Vision‑Language Model pre‑training framework that addresses the lack of fine‑grained visual perception in existing VLMs. It introduces a unified token space and a dense‑structural‑semantic vision tokenizer that expands the textual vocabulary with visual tokens, enabling vision‑language unified autoregressive supervision over both visual details and linguistic content. Trained on 12.4 T tokens, VIVAS achieves state‑of‑the‑art results on 7 tasks and 39 multimodal benchmarks.
The paper introduces a token‑oriented semantic communication framework that transmits only task‑relevant image latents instead of full token embeddings, reducing communication cost and improving interoperability. It leverages a spatial alignment between vision transformer patch tokens and learned image compression latents, enabling token‑level relevance estimation and selective transmission. Experiments on ImageNet demonstrate a superior rate–accuracy trade‑off compared to existing semantic communication methods and hand‑crafted codecs.
UVU is a vision-language unified autoregressive framework that integrates visual supervision directly into the pre-training stage of multimodal large language models. By using continuous visual encoding and a large-scale iterative hierarchical clustering algorithm to build a pixel-level visual codebook, UVU enables lossless representation of visual inputs and autoregressive generation of pixel-level image tokens alongside textual tokens. This approach synergizes pixel-level visual perception with semantic-level visual understanding, allowing models to internalize visual reconstruction capabilities and improve multimodal understanding performance.
arXiv:2608.23864v1 Announce Type: new Abstract: Visual tokenizers increasingly inject semantic supervision into latent spaces to make downstream diffusion easier. Yet how these semantics should be or...
Vision-language models commonly project all tokens produced by a pretrained vision encoder into a large language model. However, final-layer features can discard text, local attributes, and spatial relationships, while high-resolution inputs substantially increase context length and inference latency.
The paper introduces ReToK, a flexible image tokenizer that uses redundant token padding and hierarchical semantic regularization to better utilize all tokens in autoregressive image generation. By activating tail tokens more frequently and aligning early token features with a pre‑trained vision model while gradually reducing regularization, ReToK mitigates the information concentration problem of traditional nested dropout tokenizers. Experiments on ImageNet 256×256 show that ReToK outperforms both flexible and fixed‑length tokenizers in generation quality.
WithinVAE is a compact variational visual autoencoder that learns semantic structure internally using a sequential bottleneck and self‑distillation from an exponential‑moving‑average copy of itself. It achieves a four‑fold compression ratio on ImageNet‑1K, improving reconstruction metrics such as PSNR by 1.36 dB over RPiAE while using half the latent capacity. The model also boosts linear probing performance, demonstrating that semantic gains arise from self‑distillation rather than external supervision.
arXiv:2607. 00371v1 Announce Type: cross Abstract: Visual AutoRegressive modeling (VAR) has pioneered a coarse-to-fine multi-scale autoregressive generative paradigm, demonstrating strong capabilities in image generation.
arXiv:2608. 08676v1 Announce Type: cross Abstract: Semantic vision encoders have become a central visual interface for multimodal understanding and semantic conditioning in image generation.
VisionWeave introduces elastic visual representation weaving, a native capability for multimodal large language models that learns where and at what granularity to encode visual information. The method combines a gated spatial pooler for coarse representations with a granularity router that allocates content‑adaptive token usage, trained end‑to‑end on large‑scale data. Experiments on Qwen3.5‑4B and Qwen3.8‑27B show that VisionWeave can save 43.0% of tokens while preserving 98.9% of performance across eight benchmarks, and delivers significant throughput gains and latency reductions when deployed on the SGLang serving engine.
arXiv:2602.20731v2 Announce Type: replace-cross Abstract: Discrete image tokenizers provide a sequential interface for vision and multimodal models, but are typically optimized for reconstruction or...