Related Experiment Videos
Supervised neural network modeling: an empirical investigation into learning from imbalanced data with labeling
Taghi M Khoshgoftaar1, Jason Van Hulse, Amri Napolitano
1Data Mining and Machine Learning Laboratory, Department of Computer and Electrical Engineering and Computer Science, Florida Atlantic University, Boca Raton, FL 33431, USA. taghi@cse.fau.edu
IEEE Transactions on Neural Networks
|March 19, 2010
Summary
Neural networks struggle with imbalanced data and labeling errors. Data sampling may not improve performance, challenging findings from other classifiers like C4.5.
Area of Science:
- Machine Learning
- Artificial Intelligence
- Data Science
Background:
- Supervised learning algorithms, including neural networks like multilayer perceptrons (MLPs) and radial basis function networks (RBFNets), are susceptible to data imperfections.
- Class imbalance and labeling errors (class noise) significantly degrade predictive performance by hindering class distinction and leading to incorrect hypotheses.
- While these issues are often studied in isolation, their combined impact on neural network learning remains underexplored.
Purpose of the Study:
- To comprehensively investigate the combined effects of class imbalance and labeling errors on neural network algorithms.
- To evaluate the efficacy of data sampling techniques in mitigating these combined challenges for neural networks.
- To compare the behavior of neural networks under these conditions with other common classifiers.
Main Methods:
- Empirical investigation of two common neural network learning algorithms (MLPs and RBFNets).
- Systematic introduction of class noise and class imbalance to training datasets.
- Assessment of data sampling strategies for addressing class imbalance in conjunction with class noise.
- Training and evaluation of over two million models to ensure robust statistical power.
Main Results:
- Both class imbalance and labeling errors individually and combinedly impair the performance of neural network learning algorithms.
- Data sampling techniques, commonly used for class imbalance, showed limited ability to improve neural network performance when labeling errors are present.
- The detrimental effects observed with neural networks differ from those reported for other classifiers, such as C4.5.
Conclusions:
- Neural network performance is significantly degraded by the combined presence of class imbalance and labeling errors.
- Standard data sampling methods may not be effective in resolving these issues for neural networks.
- Findings highlight the need for specialized approaches when training neural networks on noisy and imbalanced datasets, as results may not generalize from other machine learning models.