FeaTok introduces feature-loss tokenization for image generation, training discrete tokens by reconstructing pretrained features instead of pixels. This approach improves generative performance across multiple self-supervised models (SigLIP, SigLIP2, DINOv2, DINOv3). In class-conditional generation, FeaTok reduces gFDr6 by up to 29% at 256×256 resolution and outperforms the strongest unguided continuous baseline with 38× less sampling compute, while also achieving higher human-preference reward-model scores and surpassing text-conditional baselines after only 500 fine-tuning updates.
Discrete image generators learn the distribution of tokens produced by an image tokenizer. These token representations should be easy for generators to model, not merely enable faithful image reconstr...
The paper introduces MTAR, a training framework for autoregressive image generation that enhances performance through multi-token prediction, token-level contrastive regularization, and semantic dropping. These components address sparse supervision, improve representation discriminability, and accelerate training without affecting inference. On ImageNet, MTAR outperforms LlamaGen with lower FID and faster training, achieving comparable results in only a third of the iterations.
The paper proposes a self‑supervised representation learning method that serves as a tokenizer for generative models. It supervises a representation loss by factorizing pointwise mutual information, estimated through differences between conditional predictions of a learned generative model. The authors demonstrate that this approach avoids dimensional collapse and feature suppression seen in methods like DINO and LeJEPA, scales predictably with compute, and achieves compute‑optimal performance over direct diffusion for pixel‑space generation, with strong scaling laws on ImageNet and GPIC.
QuadTok introduces a quadtree-based visual tokenizer that replaces traditional 2D grids or 1D token sequences with a hierarchical structure, allowing dynamic allocation of tokens to visually complex regions while using coarser resolution for homogeneous areas. The tokenizer, trained on ImageNet, reduces token usage by about 10% compared to a fixed 256-token grid and retains comparable reconstruction fidelity. Leveraging the tree’s natural causality, a 947 M GPT-style model conditioned on a quadtree topology achieves a 2.08 gFID on ImageNet 256×256 and supports zero‑shot spatially controlled image generation.
arXiv:2606. 05552v1 Announce Type: new Abstract: Despite progress in image tokenization, standard methods encode redundant information by mixing all granularities within each token, thus redundancy persists between tokens.
arXiv:2610.09242v1 Announce Type: new Abstract: Representation Autoencoders (RAEs) generate images from pre-trained visual fea- tures, but their dense token grids make generative modeling expensive....
One-dimensional (1D) variable-length visual tokenizers enable adaptive compression by varying the number of tokens, allowing downstream autoregressive (AR) models to flexibly trade off generation qual...
arXiv:2609.36756v1 Announce Type: cross Abstract: One-dimensional (1D) variable-length visual tokenizers enable adaptive compression by varying the number of tokens, allowing downstream autoregressiv...
arXiv:2407.03463v2 Announce Type: replace-cross Abstract: In the realm of self-supervised learning (SSL), conventional wisdom has gravitated towards the utility of massive, general domain datasets fo...
The paper introduces a lightweight model that learns controlled latent dynamics to accelerate masked image generation models (MIGMs). By incorporating previous features and sampled tokens, it regresses the average velocity field of feature evolution, reducing redundancy from bi-directional attention. Applied to Lumina-DiMOO, the method achieves over 4× faster text-to-image generation while preserving quality, advancing the efficiency frontier for MIGMs.
arXiv:2609.39222v1 Announce Type: new Abstract: High-compression tokenizers are essential for scaling latent image generative models. However, aggressive compression creates a fundamental tradeoff be...
arXiv:2609.24691v1 Announce Type: new Abstract: Latent diffusion models now dominate medical image generation, and every such pipeline rests on a \emph{tokenizer} that compresses images into the late...