MoDA: Modeling Deformable 3D Objects from Casual Videos

by   Chaoyue Song, et al.

In this paper, we focus on the challenges of modeling deformable 3D objects from casual videos. With the popularity of neural radiance fields (NeRF), many works extend it to dynamic scenes with a canonical NeRF and a deformation model that achieves 3D point transformation between the observation space and the canonical space. Recent works rely on linear blend skinning (LBS) to achieve the canonical-observation transformation. However, the linearly weighted combination of rigid transformation matrices is not guaranteed to be rigid. As a matter of fact, unexpected scale and shear factors often appear. In practice, using LBS as the deformation model can always lead to skin-collapsing artifacts for bending or twisting motions. To solve this problem, we propose neural dual quaternion blend skinning (NeuDBS) to achieve 3D point deformation, which can perform rigid transformation without skin-collapsing artifacts. Besides, we introduce a texture filtering approach for texture rendering that effectively minimizes the impact of noisy colors outside target deformable objects. Extensive experiments on real and synthetic datasets show that our approach can reconstruct 3D models for humans and animals with better qualitative and quantitative performance than state-of-the-art methods.


page 1

page 3

page 4

page 7

page 8


Animatable Neural Radiance Fields for Human Body Modeling

This paper addresses the challenge of reconstructing an animatable human...

NDF: Neural Deformable Fields for Dynamic Human Modelling

We propose Neural Deformable Fields (NDF), a new representation for dyna...

CaDeX: Learning Canonical Deformation Coordinate Space for Dynamic Surface Representation via Neural Homeomorphism

While neural representations for static 3D shapes are widely studied, re...

DEMEA: Deep Mesh Autoencoders for Non-Rigidly Deforming Objects

Mesh autoencoders are commonly used for dimensionality reduction, sampli...

Reduced Representation of Deformation Fields for Effective Non-rigid Shape Matching

In this work we present a novel approach for computing correspondences b...

TransHuman: A Transformer-based Human Representation for Generalizable Neural Human Rendering

In this paper, we focus on the task of generalizable neural human render...

HyperNeRF: A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fields

Neural Radiance Fields (NeRF) are able to reconstruct scenes with unprec...

Code Repositories


MoDA: Modeling Deformable 3D Objects from Casual Videos

view repo

Please sign up or login with your details

Forgot password? Click here to reset