arXiv Machine Learning

Geodesic-informed Generative Diffusion Model For Topology-preserved Image Video Generation

Hugging Face Trending Papers
Jun 29

UniGP: Taming Diffusion Transformer for Prior-Preserved Unified Generation and Perception

Recent advances in diffusion models have shown impressive performance in controllable image generation and dense prediction tasks. However, existing approaches typically treat diffusion-based controllable generation and dense prediction as separate tasks, overlooking the potential benefits of jointly modeling the heterogeneous distributions.

arXiv Computer Vision
Sep 4

Vitality-Aware Compression for Efficient Image-to-Shape Diffusion Transformers

The paper introduces a novel compression framework for image-to-shape Diffusion Transformers (DiTs) that significantly reduces model size while preserving geometric fidelity. By exploiting the non-uniform importance of 3D DiT layers, the authors combine structured pruning, adaptive quantization, and targeted fine‑tuning into a vitality‑guided approach. The method achieves up to a 66% reduction in model size across state‑of‑the‑art image‑to‑3D models without compromising synthesis quality, offering a plug‑and‑play solution for efficient 3D shape generation.

By Jaeah Lee, Hyunjin Kim, Jaewoong Cho, Gihyun Kwon
arXiv Computer Vision
Sep 4

PoseDreamer: Scalable and Photorealistic Human Data Generation Pipeline with Diffusion Models

PoseDreamer is a new pipeline that uses diffusion models to generate large‑scale synthetic datasets for 3D human mesh estimation, providing 3D mesh annotations that remain aligned with the generated images. The system incorporates controllable image generation, Direct Preference Optimization for control alignment, curriculum‑based hard sample mining, and multi‑stage quality filtering to produce over 500,000 high‑quality samples with a 76% improvement in image‑quality metrics over traditional rendering‑based datasets. Models trained on PoseDreamer match or surpass those trained on real‑world or conventional synthetic data, and combining PoseDreamer with synthetic datasets yields better performance than mixing real and synthetic data alone.

By Lorenza Prospero, Orest Kupyn, Ostap Viniavskyi, Jo\~ao F. Henriques, Christian Rupprecht
arXiv Computer Vision
Aug 31

Video Generative Models as Geometry Learner

The paper introduces GeoNeXt, a framework that repurposes pretrained video generative models for geometry estimation by framing it as a next‑frame prediction task. Unlike prior methods that either train separate depth/normal models or fine‑tune image diffusion backbones, GeoNeXt jointly models images and geometric targets, leveraging the structured knowledge of video models for more data‑efficient learning. Experiments show zero‑shot monocular depth and surface normal estimation that outperforms existing generative approaches and rivals discriminative state‑of‑the‑art methods while using far less training data.

By Haosen Yang, Jifei Song, Zhensong Zhang, Xiatian Zhu, Jiankang Deng
arXiv Computer Vision
Aug 28

SpatialCrafter: Single Image World Modeling with Generative 3D Proxies

SpatialCrafter introduces a two‑stage framework for single‑image world modeling that first generates a global 3D proxy using a Point‑anchored Sparse Structure Flow module, then refines appearance with a Generative Deferred Refiner built on a video diffusion model. The method incorporates Parallel Geometry Injection and Proxy‑Aware Corruption training to integrate the proxy without disrupting the pretrained generative manifold, and it is evaluated on a newly constructed dataset of 115K scenes. Experiments demonstrate that SpatialCrafter outperforms existing approaches, reducing long‑term drift and maintaining consistency under rapid camera motion and extreme viewpoints.

By Chuan Fang, Lingteng Qiu, Yixun Liang, Rui Chen, Kunming Luo, Zhaohua Zheng, Tongyuan Bai, Feipeng Tian, Zilong Dong, Zihan Zhou, Ping Tan