arXiv Computer Vision

UniLayDiff: A Unified Diffusion Transformer for Content-Aware Layout Generation

UniLayDiff is a Unified Diffusion Transformer that tackles content‑aware layout generation across a wide range of tasks—unconditional, element‑type, size, and relationship‑conditioned generation—using a single end‑to‑end trainable model. It treats layout constraints as a distinct modality within a Multi‑Modal Diffusion Transformer framework, capturing interactions among background images, layout elements, and constraints. The model is further refined with LoRA fine‑tuning to incorporate relation constraints, achieving state‑of‑the‑art performance and, according to the authors, the first unified solution for all content‑aware layout generation tasks.

arXiv Computer Vision

Sep 11

Learning Interaction between Image and Layout Priors for Joint Image-Layout Generation in Design Templates

The paper introduces InterIL, a joint generative model that simultaneously produces a background image and a layout of foreground elements for graphic design templates, addressing the limitations of sequential generation approaches. InterIL connects pretrained image and layout diffusion backbones via a learnable communication module, freezing the backbones to preserve prior knowledge while training only the interaction module. The model also offers a test‑time guidance strategy, enabling users to impose preferences without retraining, and demonstrates superior image, layout, and harmonization quality compared to previous methods.

By Shirong Yang, Bo Yang, Ying Cao

diffusion safety

arXiv Machine Learning

Aug 3

CompoSE: Compositional Synthesis and Editing of 3D Shapes via Part-Aware Control

arXiv:2605. 19350v2 Announce Type: replace-cross Abstract: Creating and editing high-quality 3D content remains a central challenge in computer graphics.

By Habib Slim, Shariq Farooq Bhat, Mohamed Elhoseiny, Yifan Wang, Mike Roberts

llms diffusion

arXiv AI

Jul 21

Think, Plan, Paint: Layout-Aware Reasoning for Controllable Image Generation in Unified Models

arXiv:2607. 16409v1 Announce Type: cross Abstract: Unified Multimodal Large Language Models (MLLMs) offer a promising paradigm for unifying visual understanding and generation, yet they still struggle to follow complex spatial instructions and logical constraints in controllable image generation.

By Junhao Liu, Jian-Wei Zhang, Tao Huang, Miles Yang, Zhao Zhong, Liefeng Bo

llms diffusion multimodal benchmarks safety

Hugging Face Trending Papers

Jul 7

High-Resolution Artwork Outpainting with Global Blueprint Guidance and Layout Control

Image outpainting extends an image beyond its original borders, requiring seamless style integration and globally coherent scene completion. Building on the success of diffusion models, recent methods have achieved substantial improvements in visual quality.

diffusion efficiency

arXiv Computer Vision

Aug 24

MultiCube: Compositional 3D Generation With Part-Level Semantic and Spatial Control

arXiv:2608.20448v1 Announce Type: cross Abstract: Digital 3D objects used in games and animation are often required to be compositional; that is, decomposed into semantically meaningful parts. Recent...

By Ava Pun, Kangle Deng, Yiheng Zhu, Jun-Yan Zhu, Maneesh Agrawala, Tinghui Zhou

llms diffusion

arXiv AI

Sep 1

DesignAsCode: Bridging Structural Editability and Visual Fidelity in Graphic Design Generation

arXiv:2602.17690v3 Announce Type: replace-cross Abstract: Graphic design generation demands a delicate balance between high visual fidelity and fine-grained structural editability. However, existing...

By Ziyuan Liu, Shizhao Sun, Danqing Huang, Yingdong Shi, Meisheng Zhang, Ji Li, Jingsong Yu, Jiang Bian

arXiv Machine Learning

Jun 9

AccioScene: Compositional 3D Scene Generation via Graph Diffusion and Interaction-driven Critics

arXiv:2502. 06819v2 Announce Type: replace Abstract: This paper presents a framework for generating 3D indoor scenes from text prompts.

By Yao Wei, Matteo Toso, Pietro Morerio, Changjae Oh, Michael Ying Yang, Alessio Del Bue

diffusion benchmarks

arXiv AI

Jun 4

On-the-fly Repulsion in the Contextual Space for Rich Diversity in Diffusion Transformers

arXiv:2603. 28762v2 Announce Type: replace-cross Abstract: Modern Text-to-Image (T2I) diffusion models have achieved remarkable semantic alignment, yet they often suffer from a significant lack of variety, converging on a narrow set of visual solutions for any given prompt.

By Omer Dahary, Benaya Koren, Daniel Garibi, Daniel Cohen-Or

llms diffusion multimodal safety

Hugging Face Trending Papers

Jun 23

EPEdit: Redefining Image Editing with Generative AI and User-Centric Design

The demand for image manipulation has seen a significant increase recently. Traditional tools like Photoshop and Capture One, while powerful, require considerable expertise to use effectively.

diffusion robotics fine-tuning

arXiv Computer Vision

5d ago

LynnReal-Omni: Native multi-modal Video Generation for Agentic Visual Workflows

arXiv:2609.15863v1 Announce Type: new Abstract: Video diffusion models are stochastic and hard to control: precise content often requires repeated sampling without guaranteed success, and long-horizo...

By Xiaofeng Mao, Peijia Lin, Shaohao Rui, Yibo Zhang, Haibin Wan, Weijie Ma

llms agents diffusion multimodal data-engineering

arXiv AI

Jul 1

Layout-Conditioned Autoregressive Text-to-Image Generation via Structured Masking

arXiv:2509. 12046v2 Announce Type: replace-cross Abstract: Although autoregressive (AR) models have demonstrated remarkable success in image generation, extending these models to layout-conditioned generation remains challenging due to the sparse nature of layout conditions and the risk of feature entanglement.

By Zirui Zheng, Takashi Isobe, Tong Shen, Xu Jia, Jianbin Zhao, Xiaomin Li, Mengmeng Ge, Baolu Li, Qinghe Wang, Dong Li, Dong Zhou, Yunzhi Zhuge, Huchuan Lu, Emad Barsoum

diffusion safety

arXiv AI

Sep 3

Bernini: Latent Semantic Planning for Video Diffusion

Bernini proposes a unified framework that separates semantic planning and pixel rendering for video generation and editing. An MLLM-based planner predicts target semantics in ViT embedding space, while a DiT-based renderer synthesizes pixels conditioned on this plan, text features, and source VAE features for editing. The approach introduces Segment-Aware 3D Rotary Positional Embedding and chain-of-thought reasoning, achieving state‑of‑the‑art performance on diverse video benchmarks.

By Bernini Team, Chenchen Liu, Junyi Chen, Lei Li, Lu Chi, Mingzhen Sun, Zhuoying Li, Yi Fu, Ruoyu Guo, Yiheng Wu, Ge Bai, Zehuan Yuan

llms rag diffusion multimodal benchmarks

Related stories