Invertible Neural BRDF for Object Inverse Rendering

by   Zhe Chen, et al.

We introduce a novel neural network-based BRDF model and a Bayesian framework for object inverse rendering, i.e., joint estimation of reflectance and natural illumination from a single image of an object of known geometry. The BRDF is expressed with an invertible neural network, namely, normalizing flow, which provides the expressive power of a high-dimensional representation, computational simplicity of a compact analytical model, and physical plausibility of a real-world BRDF. We extract the latent space of real-world reflectance by conditioning this model, which directly results in a strong reflectance prior. We refer to this model as the invertible neural BRDF model (iBRDF). We also devise a deep illumination prior by leveraging the structural bias of deep neural networks. By integrating this novel BRDF model and reflectance and illumination priors in a MAP estimation formulation, we show that this joint estimation can be computed efficiently with stochastic gradient descent. We experimentally validate the accuracy of the invertible neural BRDF model on a large number of measured data and demonstrate its use in object inverse rendering on a number of synthetic and real images. The results show new ways in which deep neural networks can help solve challenging radiometric inverse problems.


page 12

page 13

page 14

page 19

page 21

page 25


Inverse Rendering of Translucent Objects using Physical and Neural Renderers

In this work, we propose an inverse rendering model that estimates 3D sh...

Neural Inverse Rendering of an Indoor Scene from a Single Image

Inverse rendering aims to estimate physical scene attributes (e.g., refl...

OpenIllumination: A Multi-Illumination Dataset for Inverse Rendering Evaluation on Real Objects

We introduce OpenIllumination, a real-world dataset containing over 108K...

NDJIR: Neural Direct and Joint Inverse Rendering for Geometry, Lights, and Materials of Real Object

The goal of inverse rendering is to decompose geometry, lights, and mate...

Rotation-Equivariant Conditional Spherical Neural Fields for Learning a Natural Illumination Prior

Inverse rendering is an ill-posed problem. Previous work has sought to r...

InverseRenderNet: Learning single image inverse rendering

We show how to train a fully convolutional neural network to perform inv...

Convergence Guarantees of Overparametrized Wide Deep Inverse Prior

Neural networks have become a prominent approach to solve inverse proble...

Please sign up or login with your details

Forgot password? Click here to reset