Generalized Radiograph Representation Learning via Cross-supervision between Images and Free-text Radiology Reports

11/04/2021
by   Hong-Yu Zhou, et al.
24

Pre-training lays the foundation for recent successes in radiograph analysis supported by deep learning. It learns transferable image representations by conducting large-scale fully-supervised or self-supervised learning on a source domain. However, supervised pre-training requires a complex and labor intensive two-stage human-assisted annotation process while self-supervised learning cannot compete with the supervised paradigm. To tackle these issues, we propose a cross-supervised methodology named REviewing FreE-text Reports for Supervision (REFERS), which acquires free supervision signals from original radiology reports accompanying the radiographs. The proposed approach employs a vision transformer and is designed to learn joint representations from multiple views within every patient study. REFERS outperforms its transfer learning and self-supervised learning counterparts on 4 well-known X-ray datasets under extremely limited supervision. Moreover, REFERS even surpasses methods based on a source domain of radiographs with human-assisted structured labels. Thus REFERS has the potential to replace canonical pre-training methodologies.

READ FULL TEXT
research
12/23/2021

SLIP: Self-supervision meets Language-Image Pre-training

Recent work has shown that self-supervised pre-training leads to improve...
research
09/04/2021

Improving Joint Learning of Chest X-Ray and Radiology Report by Word Region Alignment

Self-supervised learning provides an opportunity to explore unlabeled ch...
research
04/18/2020

Self-Supervised Representation Learning on Document Images

This work analyses the impact of self-supervised pre-training on documen...
research
12/05/2019

Self-Supervised Contextual Language Representation of Radiology Reports to Improve the Identification of Communication Urgency

Machine learning methods have recently achieved high-performance in biom...
research
06/17/2021

An Evaluation of Self-Supervised Pre-Training for Skin-Lesion Analysis

Self-supervised pre-training appears as an advantageous alternative to s...
research
05/04/2023

Self-Supervised Learning for Organs At Risk and Tumor Segmentation with Uncertainty Quantification

In this study, our goal is to show the impact of self-supervised pre-tra...
research
01/05/2023

MedKLIP: Medical Knowledge Enhanced Language-Image Pre-Training

In this paper, we consider the problem of enhancing self-supervised visu...

Please sign up or login with your details

Forgot password? Click here to reset