arXiv AI
Taming the Entropy Cliff: Variable Codebook Size Quantization for Autoregressive Visual Generation
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Computer Vision
Sep 23
StableVQ: Practical Guidelines for Stable Vector-Quantized Tokenizer Training
StableVQ introduces practical guidelines to improve training stability for vector‑quantized tokenizers used in image generation models. It addresses instability caused by the entanglement of encoder–decoder and codebook training by proposing three techniques: Dynamic STE for the encoder, Region VQ Loss for the codebook, and a Decoupled Schedule for independent learning rates. Experiments on ImageNet show consistent gains in stability, codebook utilization, and reconstruction quality across various settings.
arXiv Machine Learning
Jul 7
LGQ: Learnable Geometric Quantization for Image Tokenization
arXiv:2602. 16086v3 Announce Type: replace-cross Abstract: Recent collapse-free quantizers such as FSQ achieve stable training by replacing the learnable codebook with an engineered geometry: a fixed scalar grid whose structure is dictated by the codebook size K.
arXiv Machine Learning
Jul 9
Fractal KV-Cache Archives: Lossless Symbolic Storage with In-Place Retrieval for Long-Context LLM Inference
arXiv:2607. 07144v1 Announce Type: new Abstract: The key-value (KV) cache dominates the memory cost of long-context autoregressive inference, and a growing body of work compresses it through quantization, eviction, or offloading.
arXiv AI
Oct 15
Making Language the Low-Rate Code of Vision: Successive Refinement from Words to Pixels
arXiv:2610.16552v1 Announce Type: new Abstract: An ordered visual code lets a short prefix stand in for an image, and the training loss decides what that prefix contains. Under pixel reconstruction t...
arXiv Machine Learning
Jul 7
Variable Bit-width Quantization: Learning Per-Group Precision for "Bigger-but-Smaller" Language Models
arXiv:2607. 02893v1 Announce Type: new Abstract: Low-bit quantization shrinks language models but treats precision as a single global hyper-parameter: every weight uses the same bit-width.
arXiv AI
Jun 16
Improved Baselines with Representation Autoencoders
arXiv:2605. 18324v2 Announce Type: replace-cross Abstract: Representation Autoencoders (RAE) replace traditional VAE with pretrained vision encoders.