Multi-Objective Self-Adaptive Particle Swarm Optimization for Large-Scale Feature Selection in Classification.

Area of Science:

• Machine Learning
• Optimization Algorithms
• Data Science

Background:

• Feature selection (FS) is crucial for enhancing machine learning performance on high-dimensional datasets.
• Existing multi-objective evolutionary algorithms (MOEAs) struggle with local optima stagnation in large-scale multi-objective FS problems (LSMOFSPs) due to expanding solution spaces and numerous irrelevant features.
• Current MOEAs often use a single candidate solution generation strategy (CSGS), which is inefficient for diverse LSMOFSPs, and parameter tuning is time-consuming.

Purpose of the Study:

• To address the limitations of existing MOEAs in LSMOFSPs.
• To propose a novel algorithm that efficiently handles large-scale multi-objective feature selection.
• To improve the performance of learning algorithms by effectively reducing feature dimensionality and classification error.

Main Methods:

• Development of a multi-objective self-adaptive particle swarm optimization (MOSaPSO) algorithm.
• Integration of a rapid nondominated sorting approach.
• Utilization of a self-adaptive mechanism combined with five modified efficient candidate solution generation strategies (CSGSs) for generating new solutions.

Main Results:

• MOSaPSO effectively reduced the number of features across ten experimental datasets.
• The algorithm significantly lowered classification error rates on both training and test sets.
• MOSaPSO demonstrated superior performance compared to existing algorithms, with performance gains increasing with dataset dimensionality.

Conclusions:

• The proposed MOSaPSO algorithm offers an effective solution for large-scale multi-objective feature selection problems.
• MOSaPSO overcomes the challenges of local optima stagnation and inefficient search strategies faced by traditional MOEAs.
• The algorithm's self-adaptive nature and multiple CSGSs contribute to its robust performance, particularly for high-dimensional data.