SMOTE for high-dimensional class-imbalanced data.

Area of Science:

• Machine Learning
• Data Science
• Statistics

Background:

• Class-imbalanced data leads to classification bias favoring the majority class, especially in high-dimensional settings.
• Oversampling and undersampling techniques aim to balance data, with undersampling generally helpful and random oversampling ineffective.
• Synthetic Minority Oversampling TEchnique (SMOTE) is a popular oversampling method, but its performance on high-dimensional data requires further investigation.

Purpose of the Study:

• To theoretically and empirically investigate the properties of SMOTE on high-dimensional data.
• To evaluate SMOTE's effectiveness in mitigating classification bias with imbalanced, high-dimensional datasets.
• To compare SMOTE's performance against random undersampling for high-dimensional data.

Main Methods:

• Theoretical analysis of SMOTE's behavior on high-dimensional data.
• Empirical evaluation using simulated and real-world high-dimensional datasets.
• Assessment of SMOTE's impact on various classifiers, including k-NN, with and without variable selection.

Main Results:

• SMOTE generally fails to reduce majority class bias in high-dimensional data for most classifiers and is less effective than random undersampling.
• SMOTE benefits k-NN classifiers for high-dimensional data only when variable selection is performed prior to its application.
• On high-dimensional data, SMOTE preserves class means but reduces data variability and introduces inter-sample correlation.

Conclusions:

• In high-dimensional settings, k-NN classifiers using Euclidean distance benefit from SMOTE only after variable selection, with more neighbors enhancing the effect.
• Applying SMOTE to k-NN classifiers on high-dimensional data without prior variable selection is strongly discouraged due to significant bias towards the minority class.