Audio-based AI classifiers show no evidence of improved COVID-19 screening over simple symptoms checkers

by   Harry Coppock, et al.

Recent work has reported that AI classifiers trained on audio recordings can accurately predict severe acute respiratory syndrome coronavirus 2 (SARSCoV2) infection status. Here, we undertake a large scale study of audio-based deep learning classifiers, as part of the UK governments pandemic response. We collect and analyse a dataset of audio recordings from 67,842 individuals with linked metadata, including reverse transcription polymerase chain reaction (PCR) test outcomes, of whom 23,514 tested positive for SARS CoV 2. Subjects were recruited via the UK governments National Health Service Test-and-Trace programme and the REal-time Assessment of Community Transmission (REACT) randomised surveillance survey. In an unadjusted analysis of our dataset AI classifiers predict SARS-CoV-2 infection status with high accuracy (Receiver Operating Characteristic Area Under the Curve (ROCAUC) 0.846 [0.838, 0.854]) consistent with the findings of previous studies. However, after matching on measured confounders, such as age, gender, and self reported symptoms, our classifiers performance is much weaker (ROC-AUC 0.619 [0.594, 0.644]). Upon quantifying the utility of audio based classifiers in practical settings, we find them to be outperformed by simple predictive scores based on user reported symptoms.


page 16

page 27

page 29


A large-scale and PCR-referenced vocal audio dataset for COVID-19

The UK COVID-19 Vocal Audio Dataset is designed for the training and eva...

Statistical Design and Analysis for Robust Machine Learning: A Case Study from COVID-19

Since early in the coronavirus disease 2019 (COVID-19) pandemic, there h...

Coswara: A respiratory sounds and symptoms dataset for remote screening of SARS-CoV-2 infection

This paper presents the Coswara dataset, a dataset containing diverse se...

The COUGHVID crowdsourcing dataset: A corpus for the study of large-scale cough analysis algorithms

Cough audio signal classification has been successfully used to diagnose...

Virufy: A Multi-Branch Deep Learning Network for Automated Detection of COVID-19

Fast and affordable solutions for COVID-19 testing are necessary to cont...

Identifying and mitigating bias in algorithms used to manage patients in a pandemic

Numerous COVID-19 clinical decision support systems have been developed....

Fact sheet: Automatic Self-Reported Personality Recognition Track

We propose an informed baseline to help disentangle the various contextu...

Please sign up or login with your details

Forgot password? Click here to reset