Multi-modal Broad Learning System for Medical Image and Text-based Classification

Area of Science:

• Medical Imaging Analysis
• Artificial Intelligence in Healthcare
• Computer-Aided Diagnosis

Background:

• Medical image classification is crucial for computer-aided diagnosis.
• Limited data availability and enhancement methods hinder current approaches.
• Need for advanced methods to improve diagnostic accuracy.

Purpose of the Study:

• To propose a novel Multi-Modal Broad Learning System (M2-BLS) for enhanced medical image classification.
• To leverage both medical images and radiology reports for simultaneous learning.
• To address the limitations of single-data-type learning models.

Main Methods:

• Developed a Multi-Modal Broad Learning System (M2-BLS) with two subnetworks.
• Implemented simultaneous learning of medical images and radiology reports.
• Utilized a closed-form solution for efficient training without iteration.

Main Results:

• M2-BLS achieved high accuracy in medical classification tasks.
• Demonstrated significant performance improvement compared to state-of-the-art (SOTA) deep models.
• Validated effectiveness on public datasets (IU X-RAY, PEIR GROSS_895).

Conclusions:

• M2-BLS offers a powerful approach for medical image classification.
• Simultaneous learning from multi-modal data significantly boosts classification performance.
• The proposed system provides an efficient and accurate solution for computer-aided diagnosis.