Hallucinated Neural Radiance Fields in the Wild

by   Xingyu Chen, et al.

Neural Radiance Fields (NeRF) has recently gained popularity for its impressive novel view synthesis ability. This paper studies the problem of hallucinated NeRF: i.e. recovering a realistic NeRF at a different time of day from a group of tourism images. Existing solutions adopt NeRF with a controllable appearance embedding to render novel views under various conditions, but cannot render view-consistent images with an unseen appearance. To solve this problem, we present an end-to-end framework for constructing a hallucinated NeRF, dubbed as Ha-NeRF. Specifically, we propose an appearance hallucination module to handle time-varying appearances and transfer them to novel views. Considering the complex occlusions of tourism images, an anti-occlusion module is introduced to decompose the static subjects for visibility accurately. Experimental results on synthetic data and real tourism photo collections demonstrate that our method can not only hallucinate the desired appearances, but also render occlusion-free images from different views. The project and supplementary materials are available at https://rover-xingyu.github.io/Ha-NeRF/.


page 1

page 5

page 6

page 7

page 8

page 9

page 10


Cross-Ray Neural Radiance Fields for Novel-view Synthesis from Unconstrained Image Collections

Neural Radiance Fields (NeRF) is a revolutionary approach for rendering ...

Adjustable Visual Appearance for Generalizable Novel View Synthesis

We present a generalizable novel view synthesis method where it is possi...

Generalizable Neural Performer: Learning Robust Radiance Fields for Human Novel View Synthesis

This work targets at using a general deep learning framework to synthesi...

ExtremeNeRF: Few-shot Neural Radiance Fields Under Unconstrained Illumination

In this paper, we propose a new challenge that synthesizes a novel view ...

DeOccNet: Learning to See Through Foreground Occlusions in Light Fields

Background objects occluded in some views of a light field (LF) camera c...

Fast Non-Rigid Radiance Fields from Monocularized Data

3D reconstruction and novel view synthesis of dynamic scenes from collec...

Semantic-aware Occlusion Filtering Neural Radiance Fields in the Wild

We present a learning framework for reconstructing neural scene represen...

Please sign up or login with your details

Forgot password? Click here to reset