CheXclusion: Fairness gaps in deep chest X-ray classifiers.

Area of Science:

• Medical Imaging Analysis
• Artificial Intelligence in Healthcare
• Clinical Decision Support Systems

Background:

• Machine learning, particularly deep learning, shows expert-level performance in clinical tasks like medical imaging.
• Concerns exist regarding potential biases in these AI systems concerning protected patient attributes.

Purpose of the Study:

• To investigate the extent of bias in state-of-the-art deep learning classifiers used for X-ray diagnostics.
• To evaluate true positive rate (TPR) disparities across various patient subgroups defined by protected attributes.

Main Methods:

• Convolutional neural networks were trained to predict 14 diagnostic labels using three public chest X-ray datasets (MIMIC-CXR, Chest-Xray8, CheXpert) and a combined dataset.
• True positive rate (TPR) disparities were assessed across patient sex, age, race, and insurance type (as a proxy for socioeconomic status).

Main Results:

• Significant TPR disparities were observed in state-of-the-art classifiers across all datasets, clinical tasks, and patient subgroups.
• A multi-source dataset exhibited the smallest disparities, suggesting potential for bias reduction.
• TPR disparities did not significantly correlate with a subgroup's proportional disease burden.

Conclusions:

• State-of-the-art deep learning models for X-ray diagnostics exhibit biases related to protected patient attributes.
• Aggregating data from multiple sources may help mitigate these algorithmic disparities.
• Clinical decision-makers must rigorously audit AI models for fairness and bias before deployment in healthcare settings.