Global Negative Correlation Learning: A Unified Framework for Global Optimization of Ensemble Models.

Area of Science:

• Machine Learning
• Ensemble Methods
• Artificial Intelligence

Background:

• Ensemble methods are crucial in machine learning, with diversity in constituent models being key to their success.
• Existing diversity promotion techniques often involve data sampling or structural modifications of models.
• A subset of ensemble methods, including Negative Correlation Learning (NCL), explicitly incorporates diversity into individual model error functions.

Purpose of the Study:

• To analyze the Negative Correlation Learning (NCL) framework and identify its limitations.
• To propose a novel ensemble framework, Global Negative Correlation Learning (GNCL), for improved performance.
• To focus optimization on the global ensemble rather than individual component errors.

Main Methods:

• Analysis of the Negative Correlation Learning (NCL) framework, revealing its focus on minimizing individual errors rather than the global ensemble's residuals.
• Development of the Global Negative Correlation Learning (GNCL) framework, prioritizing global ensemble optimization.
• Derivation of an analytical solution for base regressor parameters within GNCL, assuming fixed basis functions, with applicability to neural networks.

Main Results:

• The study demonstrates that NCL minimizes a combination of individual errors, not the final ensemble's residuals.
• GNCL framework focuses on optimizing the global ensemble's performance.
• Extensive experiments on regression and classification datasets show GNCL outperforms state-of-the-art ensemble methods.

Conclusions:

• The proposed Global Negative Correlation Learning (GNCL) framework offers a significant advancement over existing ensemble methods.
• GNCL's focus on global optimization leads to superior performance compared to NCL and other contemporary techniques.
• The framework provides a new direction for developing high-performing ensemble models in machine learning.