Predicting Slice-to-Volume Transformation in Presence of Arbitrary Subject Motion

by   Benjamin Hou, et al.

This paper aims to solve a fundamental problem in intensity-based 2D/3D registration, which concerns the limited capture range and need for very good initialization of state-of-the-art image registration methods. We propose a regression approach that learns to predict rotation and translations of arbitrary 2D image slices from 3D volumes, with respect to a learned canonical atlas co-ordinate system. To this end, we utilize Convolutional Neural Networks (CNNs) to learn the highly complex regression function that maps 2D image slices into their correct position and orientation in 3D space. Our approach is attractive in challenging imaging scenarios, where significant subject motion complicates reconstruction performance of 3D volumes from 2D slice data. We extensively evaluate the effectiveness of our approach quantitatively on simulated MRI brain data with extreme random motion. We further demonstrate qualitative results on fetal MRI where our method is integrated into a full reconstruction and motion compensation pipeline. With our CNN regression approach we obtain an average prediction error of 7mm on simulated data, and convincing reconstruction quality of images of very young fetuses where previous methods fail. We further discuss applications to Computed Tomography and X-ray projections. Our approach is a general solution to the 2D/3D initialization problem. It is computationally efficient, with prediction times per slice of a few milliseconds, making it suitable for real-time scenarios.


page 6

page 7

page 12

page 13

page 14


3D Reconstruction in Canonical Co-ordinate Space from Arbitrarily Oriented 2D Images

Limited capture range and the requirement to provide high quality initia...

SVoRT: Iterative Transformer for Slice-to-Volume Registration in Fetal Brain MRI

Volumetric reconstruction of fetal brains from multiple stacks of MR sli...

Real-time Deep Registration With Geodesic Loss

With an aim to increase the capture range and accelerate the performance...

Efficient brain age prediction from 3D MRI volumes using 2D projections

Using 3D CNNs on high resolution medical volumes is very computationally...

Initialize globally before acting locally: Enabling Landmark-free 3D US to MRI Registration

Registration of partial-view 3D US volumes with MRI data is influenced b...

Multimodal MRI Neuroimaging with Motion Compensation Based on Particle Filtering

Head movement during scanning impedes activation detection in fMRI studi...

Please sign up or login with your details

Forgot password? Click here to reset