Constructing ensembles of classifiers by means of weighted instance selection

Area of Science:

• Machine Learning
• Ensemble Methods
• Instance Selection

Background:

• Instance selection aims to reduce training data size while maintaining classifier performance.
• Boosting methods iteratively build ensembles by focusing on difficult instances.
• Combining these methods offers a novel approach to ensemble construction.

Purpose of the Study:

• To develop a novel method for constructing classifier ensembles by integrating instance selection with boosting.
• To investigate the effectiveness of this combined approach, termed boosting by instance selection.
• To demonstrate the benefits of this method in terms of performance and complexity reduction.

Main Methods:

• Instance selection algorithms are adapted for boosting by optimizing a weighted training error.
• The proposed method is initially applied to k-nearest neighbor (k-NN) classifiers.
• The approach is extended to other classifiers like decision trees and support vector machines (SVMs).

Main Results:

• The boosting by instance selection method produces better and simpler ensembles compared to the random subspace method (RSM) for k-NN.
• It outperforms standard ensemble methods for C4.5 and SVMs.
• The method effectively reduces the space complexity of the final ensemble.

Conclusions:

• Boosting by instance selection is a viable and advantageous technique for building high-performance, کم-complexity classifier ensembles.
• The methodology shows promise for various classifier types beyond k-NN.
• This approach offers significant improvements over existing ensemble construction techniques.