Knowledge distillation for semi-supervised domain adaptation

by   Mauricio Orbes-Arteaga, et al.

In the absence of sufficient data variation (e.g., scanner and protocol variability) in annotated data, deep neural networks (DNNs) tend to overfit during training. As a result, their performance is significantly lower on data from unseen sources compared to the performance on data from the same source as the training data. Semi-supervised domain adaptation methods can alleviate this problem by tuning networks to new target domains without the need for annotated data from these domains. Adversarial domain adaptation (ADA) methods are a popular choice that aim to train networks in such a way that the features generated are domain agnostic. However, these methods require careful dataset-specific selection of hyperparameters such as the complexity of the discriminator in order to achieve a reasonable performance. We propose to use knowledge distillation (KD) -- an efficient way of transferring knowledge between different DNNs -- for semi-supervised domain adaption of DNNs. It does not require dataset-specific hyperparameter tuning, making it generally applicable. The proposed method is compared to ADA for segmentation of white matter hyperintensities (WMH) in magnetic resonance imaging (MRI) scans generated by scanners that are not a part of the training set. Compared with both the baseline DNN (trained on source domain only and without any adaption to target domain) and with using ADA for semi-supervised domain adaptation, the proposed method achieves significantly higher WMH dice scores.


Adversarial Variational Domain Adaptation

In this work we address the problem of transferring knowledge obtained f...

Large-scale ASR Domain Adaptation using Self- and Semi-supervised Learning

Self- and semi-supervised learning methods have been actively investigat...

Federated Semi-Supervised Domain Adaptation via Knowledge Transfer

Given the rapidly changing machine learning environments and expensive d...

d-SNE: Domain Adaptation using Stochastic Neighborhood Embedding

Deep neural networks often require copious amount of labeled-data to tra...

Domain Generalization for Prostate Segmentation in Transrectal Ultrasound Images: A Multi-center Study

Prostate biopsy and image-guided treatment procedures are often performe...

First U-Net Layers Contain More Domain Specific Information Than The Last Ones

MRI scans appearance significantly depends on scanning protocols and, co...

Deep Transparent Prediction through Latent Representation Analysis

The paper presents a novel deep learning approach, which extracts latent...

Please sign up or login with your details

Forgot password? Click here to reset