OligoIS: Scalable Instance Selection for Class-Imbalanced Data Sets.

Area of Science:

• Computer Science
• Data Mining
• Machine Learning

Background:

• Modern research generates vast amounts of data, posing challenges for data mining algorithms.
• Many critical research areas like bioinformatics and security face large datasets with imbalanced sample distributions.
• Existing methods for imbalanced data are often not scalable to the massive datasets encountered in these fields.

Purpose of the Study:

• To propose a novel, scalable approach for addressing the class-imbalance problem in large datasets.
• To develop a method that can handle millions of instances and hundreds of features efficiently.
• To improve the performance of instance selection methods on imbalanced data.

Main Methods:

• A divide-and-conquer strategy is employed, processing balanced subsets of the data.
• The algorithm achieves linear time execution, enhancing scalability.
• The method is designed for parallel environments and can operate without loading the entire dataset into memory.

Main Results:

• Demonstrated improvement over state-of-the-art instance selection methods on 40 imbalanced datasets.
• Validated scalability to millions of instances and hundreds of features using three very large datasets.
• The proposed method effectively handles class-imbalanced data in large-scale scenarios.

Conclusions:

• The proposed divide-and-conquer approach offers a scalable solution for class-imbalanced learning on massive datasets.
• This method enhances data mining performance in computationally intensive research areas.
• The technique is practical for implementation in parallel computing environments.