Dense Object Reconstruction from RGBD Images with Embedded Deep Shape Representations

by   Lan Hu, et al.

Most problems involving simultaneous localization and mapping can nowadays be solved using one of two fundamentally different approaches. The traditional approach is given by a least-squares objective, which minimizes many local photometric or geometric residuals over explicitly parametrized structure and camera parameters. Unmodeled effects violating the lambertian surface assumption or geometric invariances of individual residuals are encountered through statistical averaging or the addition of robust kernels and smoothness terms. Aiming at more accurate measurement models and the inclusion of higher-order shape priors, the community more recently shifted its attention to deep end-to-end models for solving geometric localization and mapping problems. However, at test-time, these feed-forward models ignore the more traditional geometric or photometric consistency terms, thus leading to a low ability to recover fine details and potentially complete failure in corner case scenarios. With an application to dense object modeling from RGBD images, our work aims at taking the best of both worlds by embedding modern higher-order object shape priors into classical iterative residual minimization objectives. We demonstrate a general ability to improve mapping accuracy with respect to each modality alone, and present a successful application to real data.


page 6

page 9

page 10


Optimizable Object Reconstruction from a Single View

3D shape reconstruction from a single image is a highly ill-posed proble...

ShapeClipper: Scalable 3D Shape Learning from Single-View Images via Geometric and CLIP-based Consistency

We present ShapeClipper, a novel method that reconstructs 3D object shap...

DirectShape: Photometric Alignment of Shape Priors for Visual Vehicle Pose and Shape Estimation

3D scene understanding from images is a challenging problem which is enc...

SC-wLS: Towards Interpretable Feed-forward Camera Re-localization

Visual re-localization aims to recover camera poses in a known environme...

Deep Level Sets: Implicit Surface Representations for 3D Shape Inference

Existing 3D surface representation approaches are unable to accurately c...

Deep-SLAM++: Object-level RGBD SLAM based on class-specific deep shape priors

In an effort to increase the capabilities of SLAM systems and produce ob...

RANSAC: Identification of Higher-Order Geometric Features and Applications in Humanoid Robot Soccer

The ability for an autonomous agent to self-localise is directly proport...

Please sign up or login with your details

Forgot password? Click here to reset