Decompose to manipulate: Manipulable Object Synthesis in 3D Medical Images with Structured Image Decomposition

by   Siqi Liu, et al.
Siemens Healthineers

The performance of medical image analysis systems is constrained by the quantity of high-quality image annotations. Such systems require data to be annotated by experts with years of training, especially when diagnostic decisions are involved. Such datasets are thus hard to scale up. In this context, it is hard for supervised learning systems to generalize to the cases that are rare in the training set but would be present in real-world clinical practices. We believe that the synthetic image samples generated by a system trained on the real data can be useful for improving the supervised learning tasks in the medical image analysis applications. Allowing the image synthesis to be manipulable could help synthetic images provide complementary information to the training data rather than simply duplicating the real-data manifold. In this paper, we propose a framework for synthesizing 3D objects, such as pulmonary nodules, in 3D medical images with manipulable properties. The manipulation is enabled by decomposing of the object of interests into its segmentation mask and a 1D vector containing the residual information. The synthetic object is refined and blended into the image context with two adversarial discriminators. We evaluate the proposed framework on lung nodules in 3D chest CT images and show that the proposed framework could generate realistic nodules with manipulable shapes, textures and locations, etc. By sampling from both the synthetic nodules and the real nodules from 2800 3D CT volumes during the classifier training, we show the synthetic patches could improve the overall nodule detection performance by average 8.44 performance metric (CPM) score.


page 3

page 6

page 7


Class-Aware Adversarial Lung Nodule Synthesis in CT Images

Though large-scale datasets are essential for training deep learning sys...

SyntheX: Scaling Up Learning-based X-ray Image Analysis Through In Silico Experiments

Artificial intelligence (AI) now enables automated interpretation of med...

Improving CNN Training using Disentanglement for Liver Lesion Classification in CT

Training data is the key component in designing algorithms for medical i...

Metrics to Quantify Global Consistency in Synthetic Medical Images

Image synthesis is increasingly being adopted in medical image processin...

Less is More: Unsupervised Mask-guided Annotated CT Image Synthesis with Minimum Manual Segmentations

As a pragmatic data augmentation tool, data synthesis has generally retu...

Please sign up or login with your details

Forgot password? Click here to reset