Attention-based Saliency Maps Improve Interpretability of Pneumothorax Classification.

Area of Science:

• Artificial Intelligence in Medical Imaging
• Deep Learning for Radiology
• Computer-Aided Diagnosis

Background:

• Chest radiograph interpretation is crucial for diagnosing thoracic conditions.
• Vision transformers (ViTs) are emerging deep learning models with potential in medical image analysis.
• Interpretability of deep learning models is essential for clinical trust and adoption.

Purpose of the Study:

• To evaluate the performance of ViTs in chest radiograph classification.
• To assess the interpretability of attention-based saliency maps generated by ViTs.
• To compare ViT performance and interpretability with traditional convolutional neural networks (CNNs) for pneumothorax classification.

Main Methods:

• Fine-tuning ViTs on large public chest radiograph datasets (CheXpert, Chest X-Ray 14, MIMIC CXR, VinBigData).
• Generating saliency maps using transformer multimodal explainability and GradCAM.
• Evaluating classification performance using Area Under the Receiver Operating Characteristic Curve (AUC) on multiple datasets.
• Conducting a user study with radiologists to assess saliency map usefulness.

Main Results:

• ViTs achieved comparable AUCs to state-of-the-art CNNs across evaluated datasets.
• Both ViT and CNN models exhibited bias towards pneumothorax tubes in saliency maps.
• Attention-based saliency maps were rated useful by 47% of radiologists, outperforming GradCAM (39%).
• Attention-based methods demonstrated superior performance across all interpretability metrics compared to GradCAM.

Conclusions:

• ViTs offer comparable classification performance to CNNs in chest radiograph analysis.
• Attention-based saliency maps from ViTs provide valuable clinical insights and are preferred by radiologists.
• ViTs and their interpretability methods represent a promising advancement in medical image analysis for thoracic diseases.