Action Learning for 3D Point Cloud Based Organ Segmentation

by   Xia Zhong, et al.

We propose a novel point cloud based 3D organ segmentation pipeline utilizing deep Q-learning. In order to preserve shape properties, the learning process is guided using a statistical shape model. The trained agent directly predicts piece-wise linear transformations for all vertices in each iteration. This mapping between the ideal transformation for an object outline estimation is learned based on image features. To this end, we introduce aperture features that extract gray values by sampling the 3D volume within the cone centered around the associated vertex and its normal vector. Our approach is also capable of estimating a hierarchical pyramid of non rigid deformations for multi-resolution meshes. In the application phase, we use a marginal approach to gradually estimate affine as well as non-rigid transformations. We performed extensive evaluations to highlight the robust performance of our approach on a variety of challenge data as well as clinical data. Additionally, our method has a run time ranging from 0.3 to 2.7 seconds to segment each organ. In addition, we show that the proposed method can be applied to different organs, X-ray based modalities, and scanning protocols without the need of transfer learning. As we learn actions, even unseen reference meshes can be processed as demonstrated in an example with the Visible Human. From this we conclude that our method is robust, and we believe that our method can be successfully applied to many more applications, in particular, in the interventional imaging space.


page 5

page 8


PWR-Align: Leveraging Part-Whole Relationships for Part-wise Rigid Point Cloud Registration in Mixed Reality Applications

We present an efficient and robust point cloud registration (PCR) workfl...

Can point cloud networks learn statistical shape models of anatomies?

Statistical Shape Modeling (SSM) is a valuable tool for investigating an...

Point Cloud Augmentation with Weighted Local Transformations

Despite the extensive usage of point clouds in 3D vision, relatively lim...

Deep Closest Point: Learning Representations for Point Cloud Registration

Point cloud registration is a key problem for computer vision applied to...

Robustness Certification for Point Cloud Models

The use of deep 3D point cloud models in safety-critical applications, s...

Morphing of Triangular Meshes in Shape Space

We present a novel approach to morph between two isometric poses of the ...

Please sign up or login with your details

Forgot password? Click here to reset