From NeRFLiX to NeRFLiX++: A General NeRF-Agnostic Restorer Paradigm

by   Kun Zhou, et al.

Neural radiance fields (NeRF) have shown great success in novel view synthesis. However, recovering high-quality details from real-world scenes is still challenging for the existing NeRF-based approaches, due to the potential imperfect calibration information and scene representation inaccuracy. Even with high-quality training frames, the synthetic novel views produced by NeRF models still suffer from notable rendering artifacts, such as noise and blur. To address this, we propose NeRFLiX, a general NeRF-agnostic restorer paradigm that learns a degradation-driven inter-viewpoint mixer. Specially, we design a NeRF-style degradation modeling approach and construct large-scale training data, enabling the possibility of effectively removing NeRF-native rendering artifacts for deep neural networks. Moreover, beyond the degradation removal, we propose an inter-viewpoint aggregation framework that fuses highly related high-quality training images, pushing the performance of cutting-edge NeRF models to entirely new levels and producing highly photo-realistic synthetic views. Based on this paradigm, we further present NeRFLiX++ with a stronger two-stage NeRF degradation simulator and a faster inter-viewpoint mixer, achieving superior performance with significantly improved computational efficiency. Notably, NeRFLiX++ is capable of restoring photo-realistic ultra-high-resolution outputs from noisy low-resolution NeRF-rendered views. Extensive experiments demonstrate the excellent restoration ability of NeRFLiX++ on various novel view synthesis benchmarks.


page 2

page 3

page 4

page 9

page 11

page 12

page 14


NeRFLiX: High-Quality Neural View Synthesis by Learning a Degradation-Driven Inter-viewpoint MiXer

Neural radiance fields (NeRF) show great success in novel view synthesis...

Control-NeRF: Editable Feature Volumes for Scene Rendering and Manipulation

We present a novel method for performing flexible, 3D-aware image conten...

StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis

We propose StyleNeRF, a 3D-aware generative model for photo-realistic hi...

MirrorNeRF: One-shot Neural Portrait RadianceField from Multi-mirror Catadioptric Imaging

Photo-realistic neural reconstruction and rendering of the human portrai...

MirrorNeRF: One-shot Neural Portrait Radiance Field from Multi-mirror Catadioptric Imaging

Photo-realistic neural reconstruction and rendering of the human portrai...

RIDCP: Revitalizing Real Image Dehazing via High-Quality Codebook Priors

Existing dehazing approaches struggle to process real-world hazy images ...

Viewpoint Selection for Photographing Architectures

This paper studies the problem of how to choose good viewpoints for taki...

Please sign up or login with your details

Forgot password? Click here to reset