DICOM LUT is a Key Step in Medical Image Preprocessing Towards AI Generalizability

Area of Science:

• Medical Imaging
• Artificial Intelligence in Medicine
• Radiology

Background:

• Deep learning models are increasingly used in medical image analysis, particularly for chest X-rays (CXR).
• Image preprocessing significantly influences the performance of these deep learning models.
• There is a lack of standardized DICOM preprocessing methods, leading to variability in model training and evaluation.

Purpose of the Study:

• To investigate the impact of histogram equalization (HE) and values-of-interest look-up-table (VOI-LUT) transformations on deep learning classifier performance for CXR.
• To evaluate how different preprocessing techniques affect model generalizability and identify potential issues like overfitting and shortcut learning.

Main Methods:

• Generated four distinct CXR datasets: raw pixel, standard DICOM processed, and both enhanced with HE.
• Trained four independent deep learning models for pneumothorax diagnosis on these datasets.
• Evaluated model performance on two external validation datasets to assess generalizability.

Main Results:

• Histogram equalization significantly impacted model performance, especially generalizability.
• Models trained solely on HE-enhanced datasets showed poorer performance on external validation sets, indicating overfitting and information loss.
• Models trained on HE-enhanced data exhibited shortcut learning, relying on spurious correlations.

Conclusions:

• Awareness of preprocessing techniques and their impact on model performance is crucial for machine learning practitioners.
• Standardized preprocessing information should be included when sharing medical imaging datasets.
• Using pixel values closer to clinical standards during dataset curation can improve model robustness and reduce information loss.