Investigating the Robustness of Vision Transformers against Label Noise in Medical Image Classification

Area of Science:

• Deep learning for medical image analysis
• Computer vision in healthcare
• Robustness of artificial intelligence models

Background:

• Label noise in medical datasets degrades deep learning model performance.
• Convolutional Neural Networks (CNNs) have been standard for medical image classification.
• Vision Transformers (ViT) offer potential for improved feature learning, especially with large datasets.

Purpose of the Study:

• To investigate the architectural robustness of Vision Transformers (ViT) against label noise in medical image classification.
• To compare the performance of ViT-based models against CNN-based models under varying label noise conditions.
• To determine the impact of pretraining on ViT's robustness to label noise.

Main Methods:

• Utilized two medical image classification datasets: COVID-DU-Ex and NCT-CRC-HE-100K.
• Injected varying rates of label noise into the datasets to simulate real-world imperfections.
• Evaluated and compared the performance of ViT and CNN architectures on these noisy datasets.
• Investigated the effect of pretraining strategies on ViT model robustness.

Main Results:

• Vision Transformers (ViT) demonstrate enhanced architectural robustness against label noise compared to CNNs.
• Model performance degrades with increasing label noise rates for both architectures, but ViTs show better resilience.
• Pretraining significantly improves the robustness of ViT models when trained on noisy supervised medical image datasets.

Conclusions:

• Vision Transformers (ViT) present a more robust alternative to CNNs for medical image classification tasks affected by label noise.
• Pretraining is a critical factor for maximizing the benefits of ViT architectures in handling noisy medical imaging data.
• Future research should explore ViT's potential in other challenging medical AI applications.