arXiv Computer Vision

UniLayDiff: A Unified Diffusion Transformer for Content-Aware Layout Generation

UniLayDiff is a Unified Diffusion Transformer that tackles content‑aware layout generation across a wide range of tasks—unconditional, element‑type, size, and relationship‑conditioned generation—using a single end‑to‑end trainable model. It treats layout constraints as a distinct modality within a Multi‑Modal Diffusion Transformer framework, capturing interactions among background images, layout elements, and constraints. The model is further refined with LoRA fine‑tuning to incorporate relation constraints, achieving state‑of‑the‑art performance and, according to the authors, the first unified solution for all content‑aware layout generation tasks.

arXiv Computer Vision
Sep 11

Learning Interaction between Image and Layout Priors for Joint Image-Layout Generation in Design Templates

The paper introduces InterIL, a joint generative model that simultaneously produces a background image and a layout of foreground elements for graphic design templates, addressing the limitations of sequential generation approaches. InterIL connects pretrained image and layout diffusion backbones via a learnable communication module, freezing the backbones to preserve prior knowledge while training only the interaction module. The model also offers a test‑time guidance strategy, enabling users to impose preferences without retraining, and demonstrates superior image, layout, and harmonization quality compared to previous methods.

By Shirong Yang, Bo Yang, Ying Cao
arXiv AI
Jul 21

Think, Plan, Paint: Layout-Aware Reasoning for Controllable Image Generation in Unified Models

arXiv:2607. 16409v1 Announce Type: cross Abstract: Unified Multimodal Large Language Models (MLLMs) offer a promising paradigm for unifying visual understanding and generation, yet they still struggle to follow complex spatial instructions and logical constraints in controllable image generation.

By Junhao Liu, Jian-Wei Zhang, Tao Huang, Miles Yang, Zhao Zhong, Liefeng Bo
arXiv Computer Vision
Oct 19

MOSS-Image Technical Report

MOSS-Image is an open-weight image generation system that combines a 5B single-stream diffusion transformer with an intention-aware prompt enhancement module to produce high-fidelity, text-rich images with precise layout control. The system supports bilingual long-form text rendering and introduces evaluation benchmarks LongLongTextBench and RealLayoutArena. It achieves state‑of‑the‑art performance on LLTB and ranks highly on RLA, with plans to release the model, training recipe, and evaluation framework for the community.

By MOSI. AI
arXiv Computer Vision
Oct 22

Sphinx: Efficiently Serving Novel View Synthesis using Regression-Guided Selective Refinement

Sphinx is a training‑free hybrid inference system for novel view synthesis that combines fast regression models with selective diffusion refinement. It first generates coarse views via regression, then applies diffusion only where needed to improve quality while reducing computation. An SLO‑aware scheduler dynamically allocates GPU resources, achieving up to 2.7× faster inference with less than 10% perceptual loss and up to 10× higher SLO attainment compared to FCFS scheduling.

By Yuchen Xia, Souvik Kundu, Mosharaf Chowdhury, Nishil Talati
arXiv Computer Vision
4d ago

Autoregressive Diffusion for Compositional Text-to-Scene Generation

The paper introduces a new sequential text‑to‑scene generation framework that jointly produces scene layouts and objects. It proposes a 3D Autoregressive Diffusion model (3D‑ARD+) that first generates coarse 3D latents conditioned on text and the current scene, then refines them into fine‑grained object geometry and appearance. A large dataset of 230K indoor scenes with paired text instructions is curated for training, and the model demonstrates the ability to place objects according to complex spatial and semantic text prompts.

By Zhenggang Tang, Yuehao Wang, Yuchen Fan, Jun-Kun Chen, Yu-Ying Yeh, Kihyuk Sohn, Zhangyang Wang, Qixing Huang, Alexander Schwing, Rakesh Ranjan, Dilin Wang, Zhicheng Yan