Deep Graph Learning for Spatially-Varying Indoor Lighting Prediction

by   Jiayang Bai, et al.

Lighting prediction from a single image is becoming increasingly important in many vision and augmented reality (AR) applications in which shading and shadow consistency between virtual and real objects should be guaranteed. However, this is a notoriously ill-posed problem, especially for indoor scenarios, because of the complexity of indoor luminaires and the limited information involved in 2D images. In this paper, we propose a graph learning-based framework for indoor lighting estimation. At its core is a new lighting model (dubbed DSGLight) based on depth-augmented Spherical Gaussians (SG) and a Graph Convolutional Network (GCN) that infers the new lighting representation from a single LDR image of limited field-of-view. Our lighting model builds 128 evenly distributed SGs over the indoor panorama, where each SG encoding the lighting and the depth around that node. The proposed GCN then learns the mapping from the input image to DSGLight. Compared with existing lighting models, our DSGLight encodes both direct lighting and indirect environmental lighting more faithfully and compactly. It also makes network training and inference more stable. The estimated depth distribution enables temporally stable shading and shadows under spatially-varying lighting. Through thorough experiments, we show that our method obviously outperforms existing methods both qualitatively and quantitatively.


page 1

page 3

page 6

page 7


Fast Spatially-Varying Indoor Lighting Estimation

We propose a real-time method to estimate spatiallyvarying indoor lighti...

Spatiotemporally Consistent HDR Indoor Lighting Estimation

We propose a physically-motivated deep learning framework to solve a gen...

Local-to-Global Panorama Inpainting for Locale-Aware Indoor Lighting Prediction

Predicting panoramic indoor lighting from a single perspective image is ...

Sparkle Vision: Seeing the World through Random Specular Microfacets

In this paper, we study the problem of reproducing the world lighting fr...

PointAR: Efficient Lighting Estimation for Mobile Augmented Reality

We propose an efficient lighting estimation pipeline that is suitable to...

StyleLight: HDR Panorama Generation for Lighting Estimation and Editing

We present a new lighting estimation and editing framework to generate h...

IllumiNet: Transferring Illumination from Planar Surfaces to Virtual Objects in Augmented Reality

This paper presents an illumination estimation method for virtual object...

Please sign up or login with your details

Forgot password? Click here to reset