Improved medical image modality classification using a combination of visual and textual features.

Area of Science:

• Computer Science
• Medical Imaging
• Machine Learning

Background:

• Medical modality classification is crucial for organizing and retrieving medical images.
• Previous approaches often relied on limited feature sets, impacting classification accuracy.
• ImageCLEF competitions provide standardized datasets for evaluating such tasks.

Purpose of the Study:

• To present an effective approach for medical image modality classification using diverse features.
• To evaluate the performance of various visual and textual features, including combinations.
• To achieve state-of-the-art results on the ImageCLEF modality classification datasets.

Main Methods:

• Utilized datasets from ImageCLEF competitions (2011-2013) for modality classification.
• Extracted four types of visual features: Local Binary Patterns, Color and Edge Directivity, Fuzzy Color and Texture Histogram, and Scale-Invariant Feature Transform (SIFT) and opponentSIFT.
• Employed a standard bag-of-words model with TF-IDF weighting for textual features.

Main Results:

• Scale-Invariant Feature Transform (SIFT) and opponentSIFT were identified as the top-performing visual features.
• Low-level fusion of visual features significantly improved classifier performance by capturing diverse image aspects.
• Incorporating textual features further enhanced predictive accuracy, leading to superior classification outcomes.

Conclusions:

• The combination of SIFT/opponentSIFT features and feature fusion offers a robust method for medical modality classification.
• Integrating both visual and textual information provides a more comprehensive image representation, boosting performance.
• The proposed approach achieved the best reported results on the evaluated ImageCLEF databases to date.