RepFusion leverages multimodal large language models (MLLMs) as noisy representation encoders to condition a diffusion transformer for denoising in representation space. By reusing MLLM outputs across denoising steps, the method achieves faster inference while maintaining performance comparable to per-step recomputation. Experiments show RepFusion outperforms baselines with similar inference budgets, demonstrating that MLLMs provide strong priors for denoising visual representations.
By Xichen Pan, Aashu Singh, Satya Narayan Shukla, Xiangjun Fan, Shlok Kumar Mishra, Saining Xie
Visual Information-Guided Parallel Decoding for Diffusion Multimodal Large Language Models introduces the VIG‑Sampler, a method that prioritizes tokens for decoding based on their attention to image tokens and penalizes redundancy in image‑attention distributions. The approach aims to improve the quality of multimodal generation by selecting more informative tokens during diffusion decoding. Experiments on seven captioning and VQA benchmarks with three open‑source dMLLMs show that VIG‑Sampler outperforms the Info‑Gain Sampler by an average of 19.3 CIDEr points and achieves better COCO Caption results using only half as many decoding steps.
By Insu Lee, Wooje Park, Wonseok Shin, Jinwoo Son, Byonghyo Shim
arXiv:2512.21004v2 Announce Type: replace
Abstract: Recent advances in pretraining general foundation models have significantly improved performance across diverse downstream tasks. While autoregress...
By Jinghan Li, Yang Jin, Hao Jiang, Yadong Mu, Yang Song, Kun Xu
arXiv:2606. 07451v1 Announce Type: cross Abstract: Vision-language models such as CLIP are highly useful for diverse tasks due to their shared image-text embedding space.
By Sweta Mahajan, Sukrut Rao, Jiahao Xie, Alexander Koller, Bernt Schiele
arXiv:2608.29335v1 Announce Type: new
Abstract: Latent generative models typically follow a two-stage pipeline, training a variational autoencoder for reconstruction and then a generative model on th...
By Guangting Zheng, Yiyuan Zhang, Tao Yang, Yunpeng Chen, Rui Zhu, Jiajun Deng, Yanyong Zhang
Traditional multimodal representation learning and generation are two stages: a contrastive or self-supervised visual encoder is trained first, followed by a separate downstream generative model. This...
The paper introduces a post‑training approach that enables a single inference process to transition from text reasoning to image synthesis, eliminating the need for explicit modality switching. Using the 14B BAGEL model, the authors demonstrate that targeted post‑training data and reward‑weighted training improve multimodal image generation across four independent T2I benchmarks. The study highlights the benefits of joint text‑image generation and strategic data selection for enhancing T2I performance.
By Jiahui Chen, Philippe Hansen-Estruch, Xiaochuang Han, Yushi Hu, Emily Dinan, Amita Kamath, Michal Drozdzal, Reyhane Askari-Hemmat, Luke Zettlemoyer, Marjan Ghazvininejad
FLAT (Flexible‑Length Aligned Transmodal representations) is a joint multimodal pre‑training framework that learns a shared encoder for images and text, producing 1‑D continuous embeddings that can be directly used by downstream generative decoders. By combining contrastive alignment with bidirectional cross‑modal generative objectives, FLAT yields representations that are both discriminative and generative, enabling cross‑modal retrieval and generation with a single pre‑training stage. The model achieves strong performance on T2I generation (GenEval 71.1), image captioning (BLEU‑4 40.5, CIDEr 138.6), and retrieval tasks (Recall@5 86.8/75.8 on MS‑COCO, 98.3/93.6 on Flickr30K), and supports linear interpolation, latent space arithmetic, and zero‑shot composed retrieval.
By Guangyu Sun, Shlok Kumar Mishra, Wentao Bao, Robert Zhenheng Yang, Xiao Wang, Xiyuan Wang, Yujunrong Ma, Chen Yuan, Max Xiangjun Fan, Jun Xiao, Jianpeng Cheng
The paper investigates how attention sparsity behaves in autoregressive image generation, finding a distinct diagonal sparsity pattern due to spatial locality of visual tokens. It introduces a diagonal‑aware sparse attention mechanism that skips KV entries along the diagonal within a recent window, achieving up to 3.1× higher throughput and 1.19× lower latency with less than 2% quality loss compared to dense inference.
By Daeun Kim, Junwha Hong, Changhun Oh, Yoonsung Kim, Yoonhyeong Lee, Jongse Park
Cola2 is a hierarchical language modeling framework that jointly optimizes continuous semantic compression, reconstruction–generation balance, and continuous–discrete co‑adaptation. It dynamically segments text into semantic spans, controls compression via negative feedback, and trains a block‑causal masked diffusion decoder with clean, noisy, and dropped latent conditions. Experiments on five downstream tasks with 210 billion training tokens show competitive scaling, and an unconverged checkpoint with 346 billion source tokens outperforms Llama‑3‑8B on HellaSwag and StoryCloze.
By Hongcan Guo, Yian Zhao, Feng Wang, Shen Nie, Guoqiang Wei, Yan Zeng, Hengshuang Zhao
Autoregressive image generation has emerged as a paradigm for multimodal AI systems due to its compatibility with transformer-based LLM serving infrastructures. However, generating thousands of visual...
MONET is an open, Apache 2.0-licensed dataset comprising 103.8 million curated image‑text pairs, sourced from 2.9 billion raw pairs. The dataset undergoes extensive safety filtering, domain filtering, duplicate removal, and re‑captioning with multiple vision‑language models, and is further enriched with synthetic samples. Each image includes pre‑computed embeddings and annotations to facilitate downstream research, and training a 4B‑parameter latent diffusion model on MONET yields competitive GenEval and DPG scores, proving its utility for large‑scale, reproducible text‑to‑image studies.
By Benjamin Aubin, Gonzalo I\~naki Quintana, Onur Tasar, Sanjeev Sreetharan, Urszula Czerwinska, Damien Henry, Cl\'ement Chadebec