On Modelling Label Uncertainty in Deep Neural Networks: Automatic Estimation of Intra-observer Variability in 2D Echocardiography Quality Assessment

by   Zhibin Liao, et al.

Uncertainty of labels in clinical data resulting from intra-observer variability can have direct impact on the reliability of assessments made by deep neural networks. In this paper, we propose a method for modelling such uncertainty in the context of 2D echocardiography (echo), which is a routine procedure for detecting cardiovascular disease at point-of-care. Echo imaging quality and acquisition time is highly dependent on the operator's experience level. Recent developments have shown the possibility of automating echo image quality quantification by mapping an expert's assessment of quality to the echo image via deep learning techniques. Nevertheless, the observer variability in the expert's assessment can impact the quality quantification accuracy. Here, we aim to model the intra-observer variability in echo quality assessment as an aleatoric uncertainty modelling regression problem with the introduction of a novel method that handles the regression problem with categorical labels. A key feature of our design is that only a single forward pass is sufficient to estimate the level of uncertainty for the network output. Compared to the 0.11 ± 0.09 absolute error (in a scale from 0 to 1) archived by the conventional regression method, the proposed method brings the error down to 0.09 ± 0.08, where the improvement is statistically significant and equivalents to 5.7% test accuracy improvement. The simplicity of the proposed approach means that it could be generalized to other applications of deep learning in medical imaging, where there is often uncertainty in clinical labels.


page 1

page 3

page 4

page 7

page 9

page 11


Bayesian approaches for Quantifying Clinicians' Variability in Medical Image Quantification

Medical imaging, including MRI, CT, and Ultrasound, plays a vital role i...

SDD-FIQA: Unsupervised Face Image Quality Assessment with Similarity Distribution Distance

In recent years, Face Image Quality Assessment (FIQA) has become an indi...

Automated Assessment of Transthoracic Echocardiogram Image Quality Using Deep Neural Networks

Standard views in two-dimensional echocardiography are well established ...

Deep Neural Network for Automatic Assessment of Dysphonia

The purpose of this work is to contribute to the understanding and impro...

An Uncertainty Aided Framework for Learning based Liver T_1ρ Mapping and Analysis

Objective: Quantitative T_1ρ imaging has potential for assessment of bio...

Prior-knowledge-informed deep learning for lacune detection and quantification using multi-site brain MRI

Lacunes of presumed vascular origin, also referred to as lacunar infarct...

Development of A Real-time POCUS Image Quality Assessment and Acquisition Guidance System

Point-of-care ultrasound (POCUS) is one of the most commonly applied too...

Please sign up or login with your details

Forgot password? Click here to reset