SceNeRFlow: Time-Consistent Reconstruction of General Dynamic Scenes

by   Edith Tretschk, et al.

Existing methods for the 4D reconstruction of general, non-rigidly deforming objects focus on novel-view synthesis and neglect correspondences. However, time consistency enables advanced downstream tasks like 3D editing, motion analysis, or virtual-asset creation. We propose SceNeRFlow to reconstruct a general, non-rigid scene in a time-consistent manner. Our dynamic-NeRF method takes multi-view RGB videos and background images from static cameras with known camera parameters as input. It then reconstructs the deformations of an estimated canonical model of the geometry and appearance in an online fashion. Since this canonical model is time-invariant, we obtain correspondences even for long-term, long-range motions. We employ neural scene representations to parametrize the components of our method. Like prior dynamic-NeRF methods, we use a backwards deformation model. We find non-trivial adaptations of this model necessary to handle larger motions: We decompose the deformations into a strongly regularized coarse component and a weakly regularized fine component, where the coarse component also extends the deformation field into the space surrounding the object, which enables tracking over time. We show experimentally that, unlike prior work that only handles small motion, our method enables the reconstruction of studio-scale motions.


page 1

page 6

page 7

page 8

page 13

page 15

page 16

page 18


Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Video

We present Non-Rigid Neural Radiance Fields (NR-NeRF), a reconstruction ...

Animatable Neural Radiance Fields for Human Body Modeling

This paper addresses the challenge of reconstructing an animatable human...

MovingParts: Motion-based 3D Part Discovery in Dynamic Radiance Field

We present MovingParts, a NeRF-based method for dynamic scene reconstruc...

Long range teleoperation for fine manipulation tasks under time-delay network conditions

We present a coarse-to-fine approach based semi-autonomous teleoperation...

Real-time Indoor Scene Reconstruction with RGBD and Inertia Input

Camera motion estimation is a key technique for 3D scene reconstruction ...

D-NeRF: Neural Radiance Fields for Dynamic Scenes

Neural rendering techniques combining machine learning with geometric re...

Fast-SNARF: A Fast Deformer for Articulated Neural Fields

Neural fields have revolutionized the area of 3D reconstruction and nove...

Please sign up or login with your details

Forgot password? Click here to reset