A twin convolutional neural network with hybrid binary optimizer for multimodal breast cancer digital image...

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computational Biology

Background:

• Deep learning excels in unimodal medical image analysis but struggles with multimodal data for complex diagnoses like breast cancer.
• Real-world breast cancer diagnosis often integrates diverse data, including mammography, MRI, and biopsy images, posing challenges for current deep learning models.
• Existing deep learning studies frequently focus on single data types, neglecting the complexities of fusing high-dimensional, heterogeneous features from multiple sources.

Purpose of the Study:

• To present a novel deep learning framework, the Twin Convolutional Neural Network (TwinCNN), for accurate breast cancer classification using multimodal data.
• To address the challenge of high-dimensional feature fusion by incorporating a binary optimization method for feature selection and a new feature fusion technique.
• To enhance the classification performance by leveraging both image features and predicted labels for multimodality analysis.

Main Methods:

• A dual/twin convolutional neural network (TwinCNN) framework was employed for modality-based feature learning at low and high levels.
• A binary optimization method was adapted to reduce feature dimensionality by eliminating non-discriminant features.
• A novel feature fusion method was developed to integrate ground-truth and predicted labels for multimodality classification.

Main Results:

• Single modalities achieved classification accuracies of 0.755 (histology) and 0.791 (mammography), with AUCs of 0.861 and 0.638, respectively.
• The fused feature method demonstrated superior performance, yielding classification accuracies of 0.977 (histology), 0.913 (mammography), and 0.667 (multimodality).
• Feature dimensionality reduction using a binary optimizer effectively removed non-discriminant features, preventing classifier bottlenecks.

Conclusions:

• Multimodal image classification, integrating image features and predicted labels, significantly improves breast cancer diagnostic performance.
• The proposed TwinCNN framework effectively handles multimodal data, outperforming unimodal approaches.
• Feature dimensionality reduction is crucial for enhancing the efficiency and accuracy of deep learning models in medical image analysis.