Cost-sensitive AdaBoost algorithm for ordinal regression based on extreme learning machine

Area of Science:

• Machine Learning
• Computer Science
• Statistics

Background:

• The Stagewise Additive Modeling using a Multiclass Exponential (SAMME) boosting algorithm is a well-established technique.
• Ordinal regression problems, characterized by naturally ordered targets, present unique challenges for standard classification algorithms.
• Existing Extreme Learning Machine (ELM) boosting techniques often require iterative dataset generation, increasing computational complexity.

Purpose of the Study:

• To extend the SAMME boosting algorithm for ordinal regression using a cost-sensitive approach.
• To integrate an Extreme Learning Machine (ELM) model as the base classifier within the enhanced boosting framework.
• To develop an efficient and unbiased method for tackling ordinal regression problems.

Main Methods:

• The proposed ensemble model utilizes an ELM with a Gaussian kernel and a regularization parameter.
• A closed-form solution for the weighted least squares problem is derived for analytical parameter estimation.
• A cost model is incorporated to weight patterns based on target order, enabling ordinal regression capabilities.

Main Results:

• The developed method analytically estimates parameters connecting hidden and output layers in each boosting iteration.
• The technique avoids the need to generate new training datasets at each iteration, unlike other ELM boosting methods.
• Experimental validation demonstrates competitive performance against existing ensemble and ELM techniques for ordinal regression.

Conclusions:

• The proposed cost-sensitive SAMME extension effectively addresses ordinal regression problems.
• The weighted least squares formulation offers an unbiased alternative to existing ELM boosting techniques.
• The method provides a computationally efficient and accurate solution for ordinal classification tasks.