Binary classification of imbalanced datasets using conformal prediction

Area of Science:

• Machine Learning
• Data Science
• Statistical Modeling

Background:

• Severely imbalanced datasets pose significant challenges for traditional modeling techniques.
• Existing methods often rely on complex or ambiguous balancing measures, potentially leading to information loss or distortion.
• Effective modeling of imbalanced data is crucial in various scientific domains, including drug discovery and bioinformatics.

Purpose of the Study:

• To evaluate Aggregated Conformal Prediction (ACP) as a robust method for modeling severely imbalanced datasets.
• To determine if additional explicit balancing measures are necessary when using the Conformal Prediction framework.
• To assess ACP's ability to identify active minority class compounds without compromising data integrity.

Main Methods:

• Implementation of the Aggregated Conformal Prediction procedure.
• Testing ACP on severely imbalanced datasets.
• Comparison with existing modeling approaches that utilize balancing measures.
• Analysis of the necessity of explicit balancing measures beyond the Conformal Prediction framework.

Main Results:

• Aggregated Conformal Prediction demonstrated effectiveness in modeling severely imbalanced datasets.
• No additional explicit balancing measures were found to be required when using ACP.
• The procedure successfully retrieved a large majority of active minority class compounds.
• Information loss or distortion was avoided during the modeling process.

Conclusions:

• Aggregated Conformal Prediction is a promising and effective approach for handling severely imbalanced datasets.
• ACP offers a simpler and less ambiguous alternative to complex balancing methods.
• The framework's ability to preserve information and identify key minority class instances makes it valuable for scientific applications.