Model-agnostic confidence measurement for aggregating multimodal ensemble models in automatic diagnostic systems.

Area of Science:

• Artificial Intelligence in Medicine
• Clinical Decision Support Systems
• Machine Learning for Healthcare

Background:

• Automatic diagnostic systems (ADSs) aid clinicians and expand healthcare access.
• Multimodal ensemble methods are effective but struggle with aggregating diverse model outputs.
• Existing methods inadequately combine probabilities from different diagnostic criteria.

Purpose of the Study:

• To develop an improved aggregation framework for multimodal ensembles in diagnostic systems.
• To address limitations in combining probabilities from heterogeneous diagnostic models.
• To enhance the accuracy and reliability of automatic disease diagnosis.

Main Methods:

• Proposed a novel aggregation framework for multimodal ensembles.
• Developed a method to transform probability scores into unified, rule-based scores.
• Implemented a model-agnostic confidence measurement using post-analysis for heterogeneous predictions.

Main Results:

• The proposed framework effectively aggregates model-agnostic confidence scores and predictions.
• Unified aggregation rule-based scores accurately reflect model outputs.
• Experimental results show superior performance compared to existing approaches.

Conclusions:

• The new framework offers a robust solution for multimodal ensemble aggregation in ADSs.
• Improved aggregation enhances the utilization of individual model strengths.
• This approach advances the development of more reliable and accurate diagnostic systems.