The paper introduces LILIE, a Language‑Informed and Lossless Image Encoder that transforms images via a normalizing flow transformer, enabling a vision‑language model to recover captions from noised features. By training JiT models with a language‑informed loss derived from LILIE, the authors achieve a 14‑point reduction in gFD on the GPIC dataset compared to a pixel‑only baseline. The code, encoders, and generative models are publicly released at https://suchiragarwal.com/lilie.
arXiv:2605.00809v3 Announce Type: replace Abstract: In this paper, we present \textbf{Gen}erative \textbf{L}anguage-\textbf{I}mage \textbf{P}re-training (GenLIP), a minimalist generative pretraining...
Visual Information-Guided Parallel Decoding for Diffusion Multimodal Large Language Models introduces the VIG‑Sampler, a method that prioritizes tokens for decoding based on their attention to image tokens and penalizes redundancy in image‑attention distributions. The approach aims to improve the quality of multimodal generation by selecting more informative tokens during diffusion decoding. Experiments on seven captioning and VQA benchmarks with three open‑source dMLLMs show that VIG‑Sampler outperforms the Info‑Gain Sampler by an average of 19.3 CIDEr points and achieves better COCO Caption results using only half as many decoding steps.
arXiv:2607. 09024v1 Announce Type: cross Abstract: Driven by next-token prediction, NLP shifted from task-specific models into powerful generalist foundation models.
arXiv:2604. 20329v3 Announce Type: replace-cross Abstract: Recent works show that image and video generators exhibit zero-shot visual understanding behaviors, in a way reminiscent of how LLMs develop emergent capabilities of language understanding and reasoning from generative pretraining.
PixelUMM is an encoder‑free model that unifies image and video understanding and generation directly in pixel space. It represents images as spatial patches and videos as spatiotemporal tubelets, feeding both through single‑layer linear projections into a shared multimodal backbone. The Mixture‑of‑Transformers architecture blends shared attention with task‑specific parameters, enabling autoregressive text prediction, pixel‑space flow matching, and clean‑pixel video generation, and experiments show competitive performance across tasks while providing design insights for future pixel‑space multimodal models.
arXiv:2512.21004v2 Announce Type: replace Abstract: Recent advances in pretraining general foundation models have significantly improved performance across diverse downstream tasks. While autoregress...
The paper proposes a self‑supervised representation learning method that serves as a tokenizer for generative models. It supervises a representation loss by factorizing pointwise mutual information, estimated through differences between conditional predictions of a learned generative model. The authors demonstrate that this approach avoids dimensional collapse and feature suppression seen in methods like DINO and LeJEPA, scales predictably with compute, and achieves compute‑optimal performance over direct diffusion for pixel‑space generation, with strong scaling laws on ImageNet and GPIC.
The paper investigates using raw, unlabeled video as a mid‑training phase for a pretrained language model. By encoding video frames into visual tokens and training the model to predict the next token, the authors mid‑train Qwen3‑1.7B on YT‑Temporal‑1B clips. The resulting model outperforms a non‑mid‑trained counterpart by 2.9 points on video benchmarks and 5.1 points on image benchmarks, while maintaining comparable text performance.
arXiv:2607. 25527v1 Announce Type: cross Abstract: Unifying visual understanding and generation in one model holds immense promise, but remains challenging and expensive due to heavy compute and data demands and conflicts between the visual features needed for these two capabilities.
arXiv:2610.09242v1 Announce Type: new Abstract: Representation Autoencoders (RAEs) generate images from pre-trained visual fea- tures, but their dense token grids make generative modeling expensive....
arXiv:2609.01147v1 Announce Type: cross Abstract: Text-rich visual inputs require models that can read, retrieve, and compress language directly in pixel space, yet existing pixel-text encoders strug...
arXiv:2606. 08847v1 Announce Type: cross Abstract: Despite the success of image generation from text descriptions, it still faces challenges that are difficult to overcome in domains such as natural language processing (NLP) and computer vision (CV).