arXiv:2505.16157v3 Announce Type: replace Abstract: Transformer-based models have made remarkable progress in image restoration (IR) tasks. However, the quadratic complexity of self-attention in Tran...
arXiv:2605. 18714v2 Announce Type: replace-cross Abstract: Unified multimodal models (UMMs) strive to consolidate visual understanding and visual generation within a single architecture.
arXiv:2608. 08676v1 Announce Type: cross Abstract: Semantic vision encoders have become a central visual interface for multimodal understanding and semantic conditioning in image generation.
arXiv:2608.14740v2 Announce Type: replace Abstract: Unified image and video creation requires a model to follow diverse instructions while preserving identity, geometry, and temporal structure from v...
arXiv:2611.05451v1 Announce Type: new Abstract: While discrete tokenization in Unified Multimodal Models (UMMs) establishes strong semantic reasoning, it introduces a strict quantization bottleneck t...
The paper introduces Vision-of-Thought (VoT), a framework that inserts a discrete visual-thinking layer between vision‑language models (VLMs) and diffusion transformers (DiTs). Instead of using VLMs solely as text encoders, they act as multimodal planners that generate VoT tokens—high‑level visual plans such as objects and layouts—before pixel rendering. A specialized VoT tokenizer is trained with a closed‑loop objective combining VLM alignment, feature reconstruction, and vector‑quantization losses, ensuring the tokens are semantically readable by the VLM while preserving necessary visual information. Experimental results show that VoT improves semantic alignment and offers a structured, interpretable interface for controllable generation.
Bernini proposes a unified framework that separates semantic planning and pixel rendering for video generation and editing. An MLLM-based planner predicts target semantics in ViT embedding space, while a DiT-based renderer synthesizes pixels conditioned on this plan, text features, and source VAE features for editing. The approach introduces Segment-Aware 3D Rotary Positional Embedding and chain-of-thought reasoning, achieving state‑of‑the‑art performance on diverse video benchmarks.
VIVAS is a new Vision‑Language Model pre‑training framework that addresses the lack of fine‑grained visual perception in existing VLMs. It introduces a unified token space and a dense‑structural‑semantic vision tokenizer that expands the textual vocabulary with visual tokens, enabling vision‑language unified autoregressive supervision over both visual details and linguistic content. Trained on 12.4 T tokens, VIVAS achieves state‑of‑the‑art results on 7 tasks and 39 multimodal benchmarks.
arXiv:2606. 13289v1 Announce Type: cross Abstract: Holistic visual tokenizers are fundamental to unified multimodal models (UMMs) as they map diverse visual inputs into a unified representation space.
QK-Edit introduces a training‑free query‑key manipulation strategy for Multimodal Diffusion Transformers (MM‑DiTs), enabling precise text‑guided image and video editing. By isolating and adjusting critical attention components, it balances prompt fidelity with structural consistency, supporting tasks such as object addition, removal, replacement, background changes, material and color adjustments, and style transformation. The method works seamlessly with state‑of‑the‑art models like FLUX and HunyuanVideo, improving editing performance without requiring additional training.
arXiv:2609.40305v1 Announce Type: new Abstract: Improving text-to-image models has traditionally relied on increasing model size or the number of denoising steps. In this work, we explore an alternat...
arXiv:2607. 19344v1 Announce Type: cross Abstract: Controllable image generation remains challenging for creative professionals, who often require precise regional control over materials, object identities, and spatial arrangements that cannot be reliably achieved through text prompting alone.