Large-scale text-to-image models are attractive backbones for dense prediction because RGB generation pretraining learns rich semantic, structural, and geometric priors. Existing generative and editing approaches reuse these priors by casting dense prediction as target generation: annotations such as depth, normals, alpha mattes, masks, and heatmaps are encoded into an RGB-trained VAE latent space and decoded back as image-like targets.
arXiv:2608.23549v1 Announce Type: new
Abstract: Rendering views using 3D scene representations such as Gaussian Splatting (3DGS), Neural Radiance Fields (NeRF), meshes, or even point clouds produces...
By Khiem Vuong, Deva Ramanan, Srinivasa Narasimhan
MoE-ViE introduces a Mixture-of-Experts vision encoder that scales efficiently for image and video understanding, outperforming dense counterparts across various sizes. The study shows fine‑grained MoE topologies provide significant gains, and proposes an auxiliary‑loss‑free balancing variant and a specialized MoE kernel to reduce inference latency. With frame‑level distillation and a novel freezing mechanism, the largest MoE‑ViE model matches state‑of‑the‑art zero‑shot performance while being 1.7× larger and 76% faster, and it outperforms other encoders when paired with a language model on both image and video benchmarks.
arXiv:2609.23153v1 Announce Type: new
Abstract: Video diffusion transformers are expensive because attention dominates long spatiotemporal token sequences. We identify the \emph{high-sparsity trap}:...
By Yuxi Liu, Haoyu Li, Zekun Zhang, Tengxu Sun, Yixiang Cai, Jiayong Li, Yifei Xia, Tianle Liu, Baole Ai, Ang Wang, Jiamang Wang, Lin Qu, Kai Zhang, Kun Yuan, Bin Cui
arXiv:2609.23169v1 Announce Type: new
Abstract: High-quality texture generation is essential for creating realistic and production-ready 3D assets. Recent multi-view diffusion methods have shown prom...
By Yibo Zhang, Ze Yuan, Nan Cao, Li Zhang, Yan-Pei Cao, Yuan-Chen Guo, Rui Ma
The paper introduces SPARK, a lightweight input‑conditioned controller that modulates only a few dominant channels in frozen Diffusion Transformer (DiT) based super‑resolution models. By predicting bounded per‑channel affine transformations for selected channels, SPARK improves reconstruction fidelity and perceptual quality without fine‑tuning the backbone or adding adapters. Experiments on three DiT‑based SR backbones across DIV2K, RealSR, and DRealSR demonstrate consistent gains while modulating only eight channels per stream and block.
By Federico Putamorsi, Leonardo Zini, Marcella Cornia, Lorenzo Baraldi
arXiv:2603.20186v2 Announce Type: replace
Abstract: In this work, we propose Image-to-Image Rectified Flow Reformulation (I2I-RFR), a practical plug-in reformulation that recasts standard I2I regress...
By Satoshi Iizuka, Shun Okamoto, Kazuhiro Fukui
Manifold4D introduces a new denoising strategy for video re‑shooting that injects a rendered point‑cloud directly into the initial noise manifold, eliminating the need for the render to be an explicit conditioning stream during denoising. This approach allows the network to rely solely on the source video as a visual condition, improving camera‑control accuracy on the DAVIS‑Traj benchmark and Vista4D set, with significant reductions in rotation and translation errors while maintaining video fidelity. User studies confirm enhanced trajectory following and dynamic consistency, especially for large yaw amplitudes and even when the render is corrupted.
By Yongqi Mao, Zijia Dai, Zhishuo Liu, Wei Xu, Kaiwei Wang, Guotao Meng
arXiv:2606. 05981v1 Announce Type: cross Abstract: Aggressive distillation of the diffusion U-Net inverts the per-frame bottleneck of real-time text-to-image pipelines: once the denoiser is a 4-step or 1-step distilled student, the text encoder becomes the critical path.
By Yoshiyuki Ootani
Recent advances in diffusion models have shown impressive performance in controllable image generation and dense prediction tasks. However, existing approaches typically treat diffusion-based controllable generation and dense prediction as separate tasks, overlooking the potential benefits of jointly modeling the heterogeneous distributions.
Large-scale visual generators are increasingly capable but costly to train, fine-tune, and deploy. We introduce Mage-Flow, a compact 4B-scale generative stack for efficient text-to-image generation and instruction-based image editing.
Rendering views using 3D scene representations such as Gaussian Splatting (3DGS), Neural Radiance Fields (NeRF), meshes, or even point clouds produces artifacts when input views are sparse or target v...