Labelling imaging datasets on the basis of neuroradiology reports: a validation study

by   David A. Wood, et al.

Natural language processing (NLP) shows promise as a means to automate the labelling of hospital-scale neuroradiology magnetic resonance imaging (MRI) datasets for computer vision applications. To date, however, there has been no thorough investigation into the validity of this approach, including determining the accuracy of report labels compared to image labels as well as examining the performance of non-specialist labellers. In this work, we draw on the experience of a team of neuroradiologists who labelled over 5000 MRI neuroradiology reports as part of a project to build a dedicated deep learning-based neuroradiology report classifier. We show that, in our experience, assigning binary labels (i.e. normal vs abnormal) to images from reports alone is highly accurate. In contrast to the binary labels, however, the accuracy of more granular labelling is dependent on the category, and we highlight reasons for this discrepancy. We also show that downstream model performance is reduced when labelling of training reports is performed by a non-specialist. To allow other researchers to accelerate their research, we make our refined abnormality definitions and labelling rules available, as well as our easy-to-use radiology report labelling app which helps streamline this process.


Automated Labelling using an Attention model for Radiology reports of MRI scans (ALARM)

Labelling large datasets for training high-capacity neural networks is a...

Deep reinforcement learning with automated label extraction from clinical reports accurately classifies 3D MRI brain volumes

Purpose: Image classification is perhaps the most fundamental task in im...

Multimodal Representation Learning of Cardiovascular Magnetic Resonance Imaging

Self-supervised learning is crucial for clinical imaging applications, g...

Expectation Maximization Pseudo Labelling for Segmentation with Limited Annotations

We study pseudo labelling and its generalisation for semi-supervised seg...

DeepSPINE: Automated Lumbar Vertebral Segmentation, Disc-level Designation, and Spinal Stenosis Grading Using Deep Learning

The high prevalence of spinal stenosis results in a large volume of MRI ...

Transfer learning with weak labels from radiology reports: application to glioma change detection

Creating large annotated datasets represents a major bottleneck for the ...

Please sign up or login with your details

Forgot password? Click here to reset