SIMBA: A robust and generalizable measure of data imbalance

Area of Science:

• Machine Learning
• Data Science
• Artificial Intelligence

Background:

• Real-world datasets in fields like health and cybersecurity often exhibit severe class imbalance.
• Data imbalance degrades machine learning model performance, leading to biased outcomes favoring majority classes.
• Assessing the true impact of data imbalance requires robust and accurate measurement techniques.

Purpose of the Study:

• To address the challenge of accurately measuring data imbalance in machine learning.
• To introduce a novel metric, the Status of Imbalance (SIMBA), that considers data distribution and overlap.
• To evaluate SIMBA's effectiveness against existing imbalance measures.

Main Methods:

• A comprehensive review of existing data imbalance measures was conducted.
• The proposed Status of Imbalance (SIMBA) metric was developed, incorporating data distribution and overlap.
• SIMBA was benchmarked against seven other imbalance measures using five machine learning models and 498 datasets (428 synthetic, 70 non-synthetic).

Main Results:

• SIMBA demonstrated superior performance in quantifying multiclass data imbalance compared to seven other measures.
• Correlation analyses between imbalance measures and classification performance confirmed SIMBA's consistency.
• Further analysis using 20 complexity measures supported SIMBA's outperformance.

Conclusions:

• The Status of Imbalance (SIMBA) is an accurate and effective measure for quantifying multiclass data imbalance.
• SIMBA's ability to consider data distribution and overlap enhances its predictive power for classification performance.
• SIMBA offers a promising solution to mitigate challenges posed by data imbalance in machine learning applications.