Multiple-instance ensemble for construction of deep heterogeneous committees for high-dimensional low-sample-size...

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Deep ensemble learning enhances neural network performance by combining multiple models.
• Committee learning, including neural network cascades, is a key area within ensemble learning.
• The high-dimensional low-sample-size (HDLS) domain presents unique challenges for traditional machine learning models.

Purpose of the Study:

• To introduce Multiple Instance Ensemble (MIE) as a novel stacking method for deep ensembles and cascades.
• To reformulate ensemble learning as a multiple-instance learning problem for improved feature representation.
• To develop and evaluate new committee learning strategies using MIE, including a concept for growing neural network cascades.

Main Methods:

• Ensemble learning process reformulated as a multiple-instance learning problem.
• Utilizing pooling operations for associating feature representations from base neural networks.
• Exploring attention mechanisms and proposing two novel committee learning strategies with MIE.
• Leveraging MIE's capability to generate pseudo-base neural networks for growing cascades.

Main Results:

• MIE provides a class of alternative ensemble methods with performance comparable to existing stacking techniques.
• Demonstrated a novel method for generating high-performing, unbounded "growing" neural network cascades.
• Verified approach across multiple HDLS datasets, achieving high performance in binary classification with low sample sizes.

Conclusions:

• MIE is an effective stacking method for deep ensemble learning, particularly in HDLS settings.
• The proposed approach offers a flexible and powerful framework for constructing advanced neural network cascades.
• MIE contributes a new perspective to ensemble learning by integrating multiple-instance learning principles.