arXiv Computer Vision By Mathias Harrer, Laura Fink, Noah Lewis, Paul Himmler, Tim Weyrich

DIRT: Depth-completed Image-based Rendering using Two-band Blending for Live Novel-view Synthesis

Read the original on arXiv Computer Vision →

The paper presents DIRT, a live novel-view synthesis method that uses depth-completed image-based rendering with a two-band blending strategy. It operates on a posed set of depth sensor observations and images, completing a sparsely filled depth buffer in screen space and fusing projected images without any neural networks or scene-specific optimization. The approach is non‑neural, training‑free, and works across heterogeneous sensors such as RGB‑D cameras, LiDAR, and thermal cameras, achieving lower latency and competitive performance in both live and conventional NVS benchmarks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

arXiv Computer Vision
Aug 24

GS-Net: Heterogeneous Vehicle Data Reuse via Generalizable Plug-and-Play 3DGS Module

GS‑Net is a lightweight plug‑and‑play module that expands sparse Structure‑from‑Motion point clouds into dense Gaussian primitives, enabling cross‑sensor view synthesis for autonomous driving. It learns a generalizable initialization for 3D Gaussian Splatting, improving rendering quality for both interpolated and extrapolated camera viewpoints. The authors introduce CARLA‑NVS, a benchmark with 12 uniformly spaced cameras, and show that GS‑Net outperforms standard 3DGS by 2.08 dB PSNR on interpolated views and 1.86 dB on extrapolated views while being 50× faster to initialize.

By Yichen Zhang, Zihan Wang, Jiali Han, Peilin Li, Jiaxun Zhang, Jianqiang Wang, Lei He, Keqiang Li
arXiv Computer Vision
Sep 3

RoGe: Novel View Synthesis via End-to-End Implicit Reconstruction and Generation

RoGe is a new end‑to‑end framework for novel view synthesis that jointly learns an implicit 3D scene representation and a video diffusion model. It eliminates the need for explicit 3D intermediates by querying the implicit scene with camera rays to produce geometric features that condition the diffusion model. Experiments on DL3DV show that RoGe surpasses reconstruction‑based, generation‑based, and hybrid baselines in image quality and temporal consistency, and ablations confirm the benefits of ray‑queried features and joint training.

By Xiaolei Lang, Ze Kang, Zehao Huang, Naiyan Wang
arXiv Computer Vision
Nov 6

Wide View Synthesis from Overlapping Perspective Cameras in Autonomous Driving

The paper introduces Wide-View Gaussian Splatting (WV‑GS), a feed‑forward framework that synthesizes geometrically consistent wide views from multiple perspective cameras in autonomous driving. WV‑GS addresses challenges such as peripheral stretching and photometric inconsistencies by using Piecewise Perspective Rendering and Spatial Affine Photometric Modeling, eliminating the need for per‑scene optimization. The authors also present WideDrive, a synthetic benchmark with dense ground‑truth wide views, and adapt a LiDAR‑projected sparse‑reference protocol for real‑world datasets, demonstrating the method’s effectiveness across multiple datasets.

By Kaixuan Zhang, Zhipeng Xiong, Minxian Li, Mingwu Ren, Xiatian Zhu
Hugging Face Trending Papers
Jul 20

FF-ProCams: Feed-Forward Gaussian Splatting for Projector-Camera System

Projector-camera (ProCams) systems achieve active scene perception and controllable appearance manipulation via structured illumination, serving as a core infrastructure for spatial augmented reality, projection mapping, and surface reflectance acquisition. Existing inverse-rendering methods for ProCams deliver high-fidelity results but rely on time-consuming per-scene optimization, while mainstream feed-forward 3D reconstruction models produce baked appearance that cannot adapt to spatially varying projector illumination.

arXiv Computer Vision
6d ago

RetinexGS: Retinex-Inspired Gaussian Splatting for Low-Light RAW Reconstruction

RetinexGS is a RAW-domain 3D Gaussian Splatting framework that improves low-light novel-view synthesis by combining Retinex-inspired appearance modeling with sensor-informed optimization. It factorizes each Gaussian’s color into a view-independent base color and a shared, position- and view-dependent modulation field, regularizing them separately to handle noisy observations. Using DNG noise metadata to weight the reconstruction loss and depth-guided initialization, RetinexGS outperforms LE3D and RawNeRF by 2.18 dB and 2.71 dB in mean sRGB PSNR across five scenes while enabling real-time rendering.

By Yulang Fei, Huaiyuan Weng, Chul Min Yeum