Classification of hyper-scale multimodal imaging datasets.

Area of Science:

• Artificial Intelligence
• Medical Imaging
• Computer Science

Background:

• Hyper-scale multi-modal datasets, containing millions of images, require efficient classification methods for diagnostic imaging data retrieval.
• Accurate modality classification of medical images is crucial for accelerating clinical outcomes and research.

Purpose of the Study:

• To demonstrate the efficacy of deep neural networks trained on a hyper-scale, multi-modal dataset for accurate medical image modality classification.
• To evaluate the performance of different Convolutional Neural Network (CNN) architectures (ResNet-50, ResNet-18, VGG16) in classifying medical imaging modalities.

Main Methods:

• A dataset of 4.5 million heterogeneous medical images was created by combining 102 diverse datasets.
• ResNet-50, ResNet-18, and VGG16 models were trained to classify images into four modalities: Computed Tomography (CT), Magnetic Resonance Imaging (MRI), Positron Emission Tomography (PET), and X-ray.
• Model performance was assessed on unseen data, measuring classification accuracy and balanced accuracy.

Main Results:

• The best performing model achieved a 96% classification accuracy on unseen data, comparable to or exceeding more complex models like EfficientNets and Vision Transformers (ViTs).
• A balanced accuracy of 86% was achieved by the top model.
• The study confirmed the feasibility of training deep learning CNNs on hyper-scale multimodal datasets.

Conclusions:

• Deep learning models trained on hyper-scale multimodal datasets can achieve high accuracy in classifying medical imaging modalities.
• These models have significant potential for real-world applications involving large-scale medical image repositories and national healthcare institutions.
• Future research could extend this classification capability to include 3D scans.