Beyond Size and Class Balance: Alpha as a New Dataset Quality Metric for Deep Learning.

Area of Science:

• Computer Science
• Machine Learning
• Medical Imaging

Background:

• High performance in deep learning for image classification relies on diverse training datasets.
• Current practices of maximizing dataset size and class balance do not ensure sufficient diversity.
• Dataset diversity is crucial for robust model performance, especially in medical imaging applications.

Purpose of the Study:

• To investigate if directly maximizing dataset diversity improves deep learning model performance.
• To introduce and evaluate ecological diversity measures for quantifying image dataset diversity.
• To compare the predictive power of diversity measures against traditional size and balance metrics.

Main Methods:

• Developed a framework of ecological diversity measures, generalizing Shannon entropy to account for image similarities.
• Analyzed thousands of subsets from seven medical imaging datasets.
• Correlated diversity measures (specifically generalized entropy, denoted as 'big alpha') with model performance metrics like balanced accuracy.

Main Results:

• Generalized entropy measures ('big alpha') were stronger predictors of performance than dataset size or class balance.
• A specific measure, $$, explained 67% of the variance in balanced accuracy, surpassing class balance (54%) and size (39%).
• Combining dataset size with $$ yielded the highest performance prediction (79%), outperforming size-plus-class-balance (74%).

Conclusions:

• Maximizing dataset diversity using ecological measures like generalized entropy is a promising strategy for enhancing deep learning performance in medical imaging.
• The proposed 'big alpha' measures offer a more effective way to assess and improve training set quality compared to traditional metrics.
• Future work should focus on implementing diversity maximization techniques to improve medical AI models.