Are Sex-based Physiological Differences the Cause of Gender Bias for Chest X-ray Diagnosis?

by   Nina Weng, et al.

While many studies have assessed the fairness of AI algorithms in the medical field, the causes of differences in prediction performance are often unknown. This lack of knowledge about the causes of bias hampers the efficacy of bias mitigation, as evidenced by the fact that simple dataset balancing still often performs best in reducing performance gaps but is unable to resolve all performance differences. In this work, we investigate the causes of gender bias in machine learning-based chest X-ray diagnosis. In particular, we explore the hypothesis that breast tissue leads to underexposure of the lungs and causes lower model performance. Methodologically, we propose a new sampling method which addresses the highly skewed distribution of recordings per patient in two widely used public datasets, while at the same time reducing the impact of label errors. Our comprehensive analysis of gender differences across diseases, datasets, and gender representations in the training set shows that dataset imbalance is not the sole cause of performance differences. Moreover, relative group performance differs strongly between datasets, indicating important dataset-specific factors influencing male/female group performance. Finally, we investigate the effect of breast tissue more specifically, by cropping out the breasts from recordings, finding that this does not resolve the observed performance gaps. In conclusion, our results indicate that dataset-specific factors, not fundamental physiological differences, are the main drivers of male–female performance gaps in chest X-ray analyses on widely used NIH and CheXpert Dataset.


page 1

page 2

page 3

page 4


Self-supervised deep convolutional neural network for chest X-ray classification

Chest radiography is a relatively cheap, widely available medical proced...

Understanding Fairness of Gender Classification Algorithms Across Gender-Race Groups

Automated gender classification has important applications in many domai...

On the Composition and Limitations of Publicly Available COVID-19 X-Ray Imaging Datasets

Machine learning based methods for diagnosis and progression prediction ...

The Resume Paradox: Greater Language Differences, Smaller Pay Gaps

Over the past decade, the gender pay gap has remained steady with women ...

Visualizing chest X-ray dataset biases using GANs

Recent work demonstrates that images from various chest X-ray datasets c...

Demonstrating The Risk of Imbalanced Datasets in Chest X-ray Image-based Diagnostics by Prototypical Relevance Propagation

The recent trend of integrating multi-source Chest X-Ray datasets to imp...

Choose Your Lenses: Flaws in Gender Bias Evaluation

Considerable efforts to measure and mitigate gender bias in recent years...

Please sign up or login with your details

Forgot password? Click here to reset