Segmentation-Renormalized Deep Feature Modulation for Unpaired Image Harmonization

by   Mengwei Ren, et al.

Deep networks are now ubiquitous in large-scale multi-center imaging studies. However, the direct aggregation of images across sites is contraindicated for downstream statistical and deep learning-based image analysis due to inconsistent contrast, resolution, and noise. To this end, in the absence of paired data, variations of Cycle-consistent Generative Adversarial Networks have been used to harmonize image sets between a source and target domain. Importantly, these methods are prone to instability, contrast inversion, intractable manipulation of pathology, and steganographic mappings which limit their reliable adoption in real-world medical imaging. In this work, based on an underlying assumption that morphological shape is consistent across imaging sites, we propose a segmentation-renormalized image translation framework to reduce inter-scanner heterogeneity while preserving anatomical layout. We replace the affine transformations used in the normalization layers within generative networks with trainable scale and shift parameters conditioned on jointly learned anatomical segmentation embeddings to modulate features at every level of translation. We evaluate our methodologies against recent baselines across several imaging modalities (T1w MRI, FLAIR MRI, and OCT) on datasets with and without lesions. Segmentation-renormalization for translation GANs yields superior image harmonization as quantified by Inception distances, demonstrates improved downstream utility via post-hoc segmentation accuracy, and improved robustness to translation perturbation and self-adversarial attacks.


page 1

page 2

page 6

page 7

page 8

page 10


Unsupervised Medical Image Translation with Adversarial Diffusion Models

Imputation of missing images via source-to-target modality translation c...

Shape-consistent Generative Adversarial Networks for multi-modal Medical segmentation maps

Image translation across domains for unpaired datasets has gained intere...

Exploring the Power of Generative Deep Learning for Image-to-Image Translation and MRI Reconstruction: A Cross-Domain Review

Deep learning has become a prominent computational modeling tool in the ...

Uncertainty-aware GAN with Adaptive Loss for Robust MRI Image Enhancement

Image-to-image translation is an ill-posed problem as unique one-to-one ...

Harmonization Across Imaging Locations(HAIL): One-Shot Learning for Brain MRI

For machine learning-based prognosis and diagnosis of rare diseases, suc...

Self-Attentive Adversarial Stain Normalization

Hematoxylin and Eosin (H&E) stained Whole Slide Images (WSIs) are utiliz...

One Model to Unite Them All: Personalized Federated Learning of Multi-Contrast MRI Synthesis

Learning-based MRI translation involves a synthesis model that maps a so...

Please sign up or login with your details

Forgot password? Click here to reset