Real-time Indoor Scene Reconstruction with RGBD and Inertia Input

by   Zunjie Zhu, et al.

Camera motion estimation is a key technique for 3D scene reconstruction and Simultaneous localization and mapping (SLAM). To make it be feasibly achieved, previous works usually assume slow camera motions, which limits its usage in many real cases. We propose an end-to-end 3D reconstruction system which combines color, depth and inertial measurements to achieve robust reconstruction with fast sensor motions. Our framework extends Kalman filter to fuse the three kinds of information and involve an iterative method to jointly optimize feature correspondences, camera poses and scene geometry. We also propose a novel geometry-aware patch deformation technique to adapt the feature appearance in image domain, leading to a more accurate feature matching under fast camera motions. Experiments show that our patch deformation method improves the accuracy of feature tracking, and our 3D reconstruction outperforms the state-of-the-art solutions under fast camera motions.


page 4

page 5


ArticulatedFusion: Real-time Reconstruction of Motion, Geometry and Segmentation Using a Single Depth Camera

This paper proposes a real-time dynamic scene reconstruction method capa...

DynaMoN: Motion-Aware Fast And Robust Camera Localization for Dynamic NeRF

Dynamic reconstruction with neural radiance fields (NeRF) requires accur...

SceNeRFlow: Time-Consistent Reconstruction of General Dynamic Scenes

Existing methods for the 4D reconstruction of general, non-rigidly defor...

A Pose-only Solution to Visual Reconstruction and Navigation

Visual navigation and three-dimensional (3D) scene reconstruction are es...

Joint axis estimation for fast and slow movements using weighted gyroscope and acceleration constraints

Sensor-to-segment calibration is a crucial step in inertial motion track...

Mono-STAR: Mono-camera Scene-level Tracking and Reconstruction

We present Mono-STAR, the first real-time 3D reconstruction system that ...

Robust Neural Routing Through Space Partitions for Camera Relocalization in Dynamic Indoor Environments

Localizing the camera in a known indoor environment is a key building bl...

Please sign up or login with your details

Forgot password? Click here to reset