Generalization Challenges in Drug-Resistant Tuberculosis Detection from Chest X-rays

Area of Science:

• Medical Imaging
• Artificial Intelligence in Medicine
• Tuberculosis Research

Background:

• Accurate classification of drug-resistant tuberculosis (DR-TB) and drug-sensitive tuberculosis (DS-TB) from chest radiographs (CXRs) is a critical unmet need.
• Previous deep convolutional neural network (CNN) models achieved 85% AUC on cross-validation but showed significant performance degradation (65% AUC) on unseen data.

Purpose of the Study:

• To investigate the generalizability of CNN models for DR-TB classification on an independent, held-out country dataset.
• To identify reasons for poor generalization, including image acquisition differences and model localization discrepancies.

Main Methods:

• Evaluated CNN model performance on unseen CXR data from a different country.
• Utilized GradCAM for model localization analysis and compared it with radiologist annotations.
• Developed a multi-country classifier to assess the impact of image acquisition variations.
• Applied a multi-task learning approach incorporating TB lesion location information.

Main Results:

• Significant performance degradation (65% AUC) was observed when generalizing to the held-out dataset.
• Model localization (GradCAM) showed limited overlap with radiologist-annotated lesion locations.
• A multi-country classifier achieved 86% accuracy in identifying the country of origin, indicating influence of image acquisition factors.
• The multi-task approach improved generalization performance on the held-out dataset to 68% AUC.

Conclusions:

• CNN models for DR-TB classification suffer from poor generalizability due to variations in image acquisition and non-pathological image features.
• Model overfitting to training data from specific countries hinders performance on unseen international datasets.
• Incorporating prior TB lesion location information via multi-task learning offers a promising strategy to enhance model generalization for DR-TB classification.