arXiv Computer Vision

RoomLight: A 2.5D Illumination Prior for Indoor Environments

arXiv Computer Vision
Sep 24

GLOW: Global Illumination-Aware Inverse Rendering of Indoor Scenes Captured with Dynamic Co-Located Light & Camera

GLOW is a Global Illumination‑aware inverse rendering framework for indoor scenes captured with dynamic co‑located light and camera setups. It combines a neural implicit surface representation with a neural radiance cache to jointly optimize geometry and reflectance, while introducing a dynamic radiance cache and a surface‑angle‑weighted radiometric loss to handle near‑field motion, strong inter‑reflections, and specular highlights. Experiments demonstrate that GLOW significantly outperforms prior methods in estimating material reflectance under both natural and co‑located illumination.

By Jiaye Wu, Saeed Hadadan, Geng Lin, Peihan Tu, Auguste Gezalyan, Matthias Zwicker, David Jacobs, Roni Sengupta
arXiv Computer Vision
Nov 6

Integrating Learned Priors through Prior-Space Reparameterization for Physically Based Inverse Rendering

The paper introduces a prior-guided material reparameterization for physically based inverse rendering, leveraging learned material predictions to restrict the optimization space without altering the photometric objective. By fusing multi-view predictions onto a mesh and constructing a permutohedral lattice in prior feature space, surface points with similar predictions share material variables, which are jointly optimized with geometry and illumination. Experiments on synthetic benchmarks and Stanford-ORB show improved material decomposition and relighting, with PSNR gains of 1.5–2.5 dB over strong baselines and a faster optimization schedule that maintains high relighting quality.

By Hun Hwangbo, Minjun Park, Yongsung Kim, Wooseok Song, Jungmin Ko, Junghyun Hwang, Sungroh Yoon
arXiv Computer Vision
Sep 11

Shedding Light: A Benchmark for Evaluating Lighting Understanding in Generative Image Models

The paper introduces a benchmark called Shedding Light to evaluate how well generative image models understand and reproduce lighting. The benchmark tests models by asking them to inpaint a simple object, called a light probe, into real photographs and then compares the generated probe to the ground truth to assess lighting direction, colour, and radiance. The authors provide a scalable protocol and open-source code and data for systematic assessment of photometric accuracy in future models.

By Justine Giroux, Jack Oliver Hilliard, Yannick Hold-Geoffroy, Javier Vazquez-Corral, Jean-Fran\c{c}ois Lalonde
arXiv AI
Sep 10

RelightFormer: Feed-forward Generative Transformer for Multiview Object Relighting

RelightFormer is a feed‑forward generative Transformer that performs single‑ and multi‑view image relighting without explicit intrinsic property estimation. It incorporates a latent illumination module that injects target environment maps into spatial features via cross‑attention, and uses permutation‑invariant positional encodings to process unordered multi‑view inputs symmetrically. Trained on the large Laval Objaverse Dataset, the model achieves state‑of‑the‑art visual and photorealistic relighting quality, and demonstrates strong zero‑shot generalization across various relighting tasks.

By Hejun Wang, Jinxi Li, Junwei Jiang, Shiwei Mao, Hu Cheng, Shouwang Huang, Bo Yang
arXiv Computer Vision
Oct 26

Do Image Editing Models Understand Lighting?

arXiv:2606.26738v3 Announce Type: replace Abstract: While recent advancements in generative image editing models have achieved stunning visual fidelity, it remains an open question whether these syst...

By Tim K\"uchler, Johann-Friedrich Feiden, Matthias Nie{\ss}ner, Carsten Rother
arXiv AI
Sep 21

Physically Based Rendering in the Latent Space

The paper introduces a method that applies physically based rendering (PBR) within the latent space of variational autoencoders used in image diffusion models. By modifying the rendering equation and using a differentiable renderer, the authors can generate latent maps that guide content creation with physically accurate lighting. The approach is trained on a single rendered image and then shown to generalize to changes in scene geometry, lighting, and camera viewpoint.

By Vuk Radovanovic, Vishesh Gupta, Adrien Gruson, Binh-Son Hua
arXiv Computer Vision
Oct 22

JointLight: Human-Centric Video Relighting with Asynchronous RGB-Irradiance Joint Modeling

JointLight introduces an asynchronous joint-modeling approach for human-centric video relighting, enabling fine-grained illumination control by jointly denoising RGB and irradiance frames during two-stage training. The method employs a domain matching strategy to bridge gaps between synthetic irradiance cues and real irradiance maps, and uses an asynchronous hijacked generation pipeline at inference. A newly created multi-illumination synthetic video dataset with paired irradiance maps supports training, and the model demonstrates strong generalization to real-world scenarios, outperforming prior methods in lighting fidelity and generative quality.

By Henglei Lv, Bailin Deng, Xiaoqiang Liu, Yichen Wang, Haoxian Zhang, Pengfei Wan, Lin Gao