ELITE: ensemble of optimal input-pruned neural networks using TRUST-TECH

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computational Neuroscience

Background:

• Neural network ensembles improve predictive accuracy and robustness.
• Constructing high-quality ensembles requires balancing accuracy and diversity.
• Existing methods face challenges in optimizing ensemble components and combinations.

Purpose of the Study:

• To develop a novel method, ELITE, for creating superior neural network ensembles.
• To enhance ensemble performance through optimal linear combination of accurate and diverse networks.
• To utilize the TRUST-TECH optimization technique for network pruning and ensemble weighting.

Main Methods:

• The ELITE method employs TRUST-TECH for global optimization, identifying multiple local optima.
• Feature selection and tier-1 TRUST-TECH search generate a diverse population of input-pruned networks.
• TRUST-TECH-based optimal training and nonlinear programming solve for optimal ensemble weights, handling non-convexity.

Main Results:

• ELITE consistently outperformed existing methods on benchmark pattern classification datasets.
• The method demonstrated effective handling of non-convexity in ensemble weight optimization.
• Numerical experiments validated the efficacy of ELITE in constructing high-quality ensembles.

Conclusions:

• The ELITE method is a promising approach for developing high-performance neural network ensembles.
• Optimal linear combination and diverse network selection are key to ELITE's success.
• TRUST-TECH provides a robust framework for optimizing ensemble components and weights.