Feature Selection Based on Intrusive Outliers Rather Than All Instances

Area of Science:

• Machine Learning
• Data Mining
• Pattern Recognition

Background:

• Traditional feature selection methods struggle with highly overlapping classes and skewed data distributions.
• Instance-based methods often overlook critical differences between outliers and the main class body, leading to misclassification.
• Identifying and utilizing specific outlier characteristics is crucial for improving classification performance.

Purpose of the Study:

• To propose a novel supervised feature selection method, Intrusive Outliers-based Feature Selection (IOFS).
• To address the challenge of misclassification caused by intrusive outliers in datasets with overlapping classes.
• To exploit the unique characteristics of intrusive outliers for more effective feature selection.

Main Methods:

• Developed a density-mean center algorithm to accurately identify class representatives and intrusive outliers (IOs).
• Introduced a specific distance threshold and mathematical formulations to evaluate the overlapping degree of intrusive class pairs.
• Proposed an extension, E-IOFS, focusing on a small subset of extreme intrusive outliers.

Main Results:

• IOFS effectively ranks features with high overlapping degrees lower, improving classification.
• Experiments on eleven benchmark datasets demonstrate IOFS's rationality and effectiveness, particularly for datasets with high class overlap.
• The extended method, E-IOFS, consistently outperformed IOFS in most experimental settings.

Conclusions:

• IOFS provides a robust approach to feature selection in the presence of challenging data distributions.
• The method successfully leverages intrusive outlier characteristics to enhance classifier performance.
• E-IOFS offers a more refined and often superior feature selection strategy by focusing on extreme outliers.