Instantiation-Net: 3D Mesh Reconstruction from Single 2D Image for Right Ventricle

by   Zhao-Yang Wang, et al.

3D shape instantiation which reconstructs the 3D shape of a target from limited 2D images or projections is an emerging technique for surgical intervention. It improves the currently less-informative and insufficient 2D navigation schemes for robot-assisted Minimally Invasive Surgery (MIS) to 3D navigation. Previously, a general and registration-free framework was proposed for 3D shape instantiation based on Kernel Partial Least Square Regression (KPLSR), requiring manually segmented anatomical structures as the pre-requisite. Two hyper-parameters including the Gaussian width and component number also need to be carefully adjusted. Deep Convolutional Neural Network (DCNN) based framework has also been proposed to reconstruct a 3D point cloud from a single 2D image, with end-to-end and fully automatic learning. In this paper, an Instantiation-Net is proposed to reconstruct the 3D mesh of a target from its a single 2D image, by using DCNN to extract features from the 2D image and Graph Convolutional Network (GCN) to reconstruct the 3D mesh, and using Fully Connected (FC) layers to connect the DCNN to GCN. Detailed validation was performed to demonstrate the practical strength of the method and its potential clinical use.


One-stage Shape Instantiation from a Single 2D Image to 3D Point Cloud

Shape instantiation which predicts the 3D shape of a dynamic target from...

STD-Net: Structure-preserving and Topology-adaptive Deformation Network for 3D Reconstruction from a Single Image

3D reconstruction from a single view image is a long-standing prob-lem i...

Monocular Human Shape and Pose with Dense Mesh-borne Local Image Features

We propose to improve on graph convolution based approaches for human sh...

Image-to-Graph Convolutional Network for Deformable Shape Reconstruction from a Single Projection Image

Shape reconstruction of deformable organs from two-dimensional X-ray ima...

A Real-time and Registration-free Framework for Dynamic Shape Instantiation

Real-time 3D navigation during minimally invasive procedures is an essen...

IGCN: Image-to-graph Convolutional Network for 2D/3D Deformable Registration

Organ shape reconstruction based on a single-projection image during tre...

Please sign up or login with your details

Forgot password? Click here to reset