Cats, not CAT scans: a study of dataset similarity in transfer learning for 2D medical image classification

by   Irma van den Brandt, et al.

Transfer learning is a commonly used strategy for medical image classification, especially via pretraining on source data and fine-tuning on target data. There is currently no consensus on how to choose appropriate source data, and in the literature we can find both evidence of favoring large natural image datasets such as ImageNet, and evidence of favoring more specialized medical datasets. In this paper we perform a systematic study with nine source datasets with natural or medical images, and three target medical datasets, all with 2D images. We find that ImageNet is the source leading to the highest performances, but also that larger datasets are not necessarily better. We also study different definitions of data similarity. We show that common intuitions about similarity may be inaccurate, and therefore not sufficient to predict an appropriate source a priori. Finally, we discuss several steps needed for further research in this field, especially with regard to other types (for example 3D) medical images. Our experiments and pretrained models are available via <>


Cats or CAT scans: transfer learning from natural or medical image source datasets?

Transfer learning is a widely used strategy in medical image analysis. I...

Medical supervised masked autoencoders: Crafting a better masking strategy and efficient fine-tuning schedule for medical image classification

Masked autoencoders (MAEs) have displayed significant potential in the c...

Bridging the gap between Natural and Medical Images through Deep Colorization

Deep learning has thrived by training on large-scale datasets. However, ...

Classification of COVID-19 in CT Scans using Multi-Source Transfer Learning

Since December of 2019, novel coronavirus disease COVID-19 has spread ar...

A Comprehensive Study of Modern Architectures and Regularization Approaches on CheXpert5000

Computer aided diagnosis (CAD) has gained an increased amount of attenti...

Contrastive Learning of Medical Visual Representations from Paired Images and Text

Learning visual representations of medical images is core to medical ima...

A scoping review of transfer learning research on medical image analysis using ImageNet

Objective: Employing transfer learning (TL) with convolutional neural ne...

Please sign up or login with your details

Forgot password? Click here to reset