arXiv AI

GroundControl: Satellite-Conditioned Multi-View Diffusion for Reconstruction-Free Novel View Synthesis

GroundControl introduces a satellite-conditioned multi‑view diffusion model for novel view synthesis that leverages both satellite imagery and street‑level inputs. The approach uses cross‑attention conditioning to preserve pretrained diffusion priors while aligning generated views with the scene layout seen from satellite images, avoiding unrealistic hallucinations. Experiments on outdoor datasets demonstrate superior performance over baselines, particularly when target views have minimal overlap with source views.

Hugging Face Trending Papers
Oct 27

GroundControl: Satellite-Conditioned Multi-View Diffusion for Reconstruction-Free Novel View Synthesis

GroundControl introduces a satellite‑conditioned multi‑view diffusion model for novel view synthesis that uses satellite images alongside street‑level inputs. The cross‑attention conditioning preserves pretrained diffusion priors while ensuring realistic spatial layouts, even when target views have little overlap with source views. Experiments on outdoor datasets demonstrate superior performance over both satellite‑only and street‑only baselines without relying on 3D reconstruction.

arXiv Computer Vision
Oct 23

MeSS: City Mesh-Guided Outdoor Scene Generation with Cross-View Consistent Diffusion

MeSS (Mesh-based Scene Synthesis) is a method that generates high‑quality, style‑consistent outdoor scenes using city mesh models as geometric priors. The pipeline first creates geometrically consistent sparse views with Cascaded Outpainting ControlNets, then densifies intermediate views via AGInpaint, and finally removes visual inconsistencies with GCAlign. Simultaneously, a 3D Gaussian Splatting scene is reconstructed from the mesh, enabling diverse rendering styles after synthesis.

By Xuyang Chen, Zhijun Zhai, Kaixuan Zhou, Zengmao Wang, Jianan He, Dong Wang, Yanfeng Zhang, Mingwei Sun, Xuqin Wang, R\"udiger Westermann, Tao Wu, Konrad Schindler, Liqiu Meng
arXiv Computer Vision
Oct 15

VGGT-Diff: Visual Geometry Meets Diffusion for Sparse-View Novel View Synthesis

arXiv:2609.33253v2 Announce Type: replace Abstract: We present VGGT-Diff, a geometry-routed multi-view diffusion model for sparse-view novel view synthesis. Existing novel view synthesis (NVS) method...

By Kangjie Chen, Xiangyu Li, Dongbin Zhang, Chaoda Zheng, Shijia Chen, Jinhao Deng, Hongbin Lin, Choo Sin Wai, Minqi Wang, Minghao Yang, Dake Zhong, Guorui Song, Yu Zhang, Xianming Liu, Boyang Wang
arXiv Computer Vision
Oct 9

LEGO: A Lifting-Free Approach for Exocentric-to-Egocentric Video Generation

The paper introduces LEGO, a lifting‑free method for generating egocentric videos from a single exocentric recording. Instead of reconstructing depth and point clouds, it trains a transformer‑based view synthesizer that directly predicts the egocentric view, providing a probabilistic correspondence map that balances structural alignment with texture averaging. This approach outperforms existing explicit pipelines and generalizes across datasets without retraining.

By Suhwan Cho, Yonwoo Choi, Soongjin Kim, Jicheol Park, Taegyu Lim
arXiv Machine Learning
Sep 14

AnyView: Synthesizing Any Novel View in Dynamic Scenes

AnyView is a diffusion-based video generation framework designed for dynamic view synthesis, requiring minimal inductive biases or geometric assumptions. It trains a generalist spatiotemporal implicit representation using diverse data sources—monocular, multi-view static, and multi-view dynamic—to produce zero-shot novel videos from arbitrary camera locations and trajectories. The authors evaluate AnyView on standard benchmarks, introduce a new challenging benchmark called AnyViewBench for extreme dynamic view synthesis, and demonstrate that AnyView outperforms baselines in maintaining realistic, plausible, and spatiotemporally consistent videos across diverse real-world scenarios.

By Basile Van Hoorick, Dian Chen, Shun Iwase, Pavel Tokmakov, Muhammad Zubair Irshad, Igor Vasiljevic, Swati Gupta, Fangzhou Cheng, Sergey Zakharov, Vitor Campagnolo Guizilini
arXiv Computer Vision
Sep 3

RoGe: Novel View Synthesis via End-to-End Implicit Reconstruction and Generation

RoGe is a new end‑to‑end framework for novel view synthesis that jointly learns an implicit 3D scene representation and a video diffusion model. It eliminates the need for explicit 3D intermediates by querying the implicit scene with camera rays to produce geometric features that condition the diffusion model. Experiments on DL3DV show that RoGe surpasses reconstruction‑based, generation‑based, and hybrid baselines in image quality and temporal consistency, and ablations confirm the benefits of ray‑queried features and joint training.

By Xiaolei Lang, Ze Kang, Zehao Huang, Naiyan Wang
arXiv Computer Vision
Oct 19

ESC: Escaping 2D Diffusion Ambiguity with Geometric Constraints for Consistent Novel View Synthesis

The paper introduces ESC, a framework that improves 3D-consistent novel view synthesis from a single 2D image by jointly generating a target view and its depth map. By conditioning the diffusion model on the source image, source depth, and target camera rays, ESC forces the model to reason in 3D even under extreme viewpoint changes. A per‑pixel depth consistency constraint is added to enforce spatial fidelity, resulting in superior visual accuracy and spatial alignment compared to current state‑of‑the‑art methods.

By Ge Ya Luo, Daniel Severo, Brian Karrer, Christopher Pal, Adriana Romero-Soriano, Michal Drozdzal