Bag of Tricks for Improving Deep Learning Performance on Multimodal Image Classification.

Area of Science:

• Artificial Intelligence in Medicine
• Medical Imaging Analysis
• Deep Learning

Background:

• Current medical image diagnosis relies on multiple modalities, often involving multi-staged processes with computational inefficiencies.
• Developing a single deep learning model capable of handling diverse medical image modalities for disease diagnosis is a significant research challenge.

Purpose of the Study:

• To present an improved end-to-end deep learning method for multimodal medical image classification.
• To explore and enhance models trained from scratch and via transfer learning for improved diagnostic performance.
• To demonstrate the model's ability to implicitly discriminate imaging modality before disease diagnosis.

Main Methods:

• Developed and evaluated end-to-end deep learning models for multimodal image classification.
• Investigated methods for training models from scratch and utilizing transfer learning.
• Applied the models to classify COVID-19 using chest X-ray, CT scan, and lung ultrasound images.

Main Results:

• The developed deep learning models demonstrated the capability to implicitly identify the imaging modality.
• The highest performing model achieved an overall accuracy of 91.07% in disease classification.
• The model successfully classified COVID-19 across chest X-ray, CT scan, and lung ultrasound modalities.

Conclusions:

• An improved end-to-end deep learning approach enables effective multimodal medical image classification.
• The model's ability to self-discriminate modality enhances diagnostic accuracy for diseases like COVID-19.
• This method offers a more efficient alternative to traditional multi-staged diagnostic processes.