Sample size planning for classification models.

Area of Science:

• Biospectroscopy
• Machine Learning
• Biomedical Data Analysis

Background:

• Statistically independent samples for classifier training and testing in biospectroscopy are scarce and expensive.
• Learning curves are essential for determining optimal training sample sizes for biospectroscopy classifiers.

Purpose of the Study:

• To address the challenge of limited sample sizes in biospectroscopy classifier development and validation.
• To determine the necessary test sample sizes for reliable performance assessment of biospectroscopy classifiers.
• To provide methods for calculating sample sizes needed to demonstrate classifier superiority.

Main Methods:

• Analysis of learning curves in small sample size scenarios (5-25 samples per class).
• Determination of required test sample sizes for precise validation (75-100 samples).
• Calculation of sample sizes for comparing classifier performance, including simulations with Raman spectra of single cells.

Main Results:

• Learning curves can be masked by testing uncertainty with small test sets.
• Approximately 75-100 independent samples are typically needed to validate a good classifier.
• Demonstrating classifier superiority often requires hundreds of samples or may be impossible.

Conclusions:

• Rigorous validation of biospectroscopy classifiers requires significantly larger test datasets than often available.
• Accurate sample size planning is crucial for reliable performance evaluation and comparison of biospectroscopy models.
• The study provides practical guidance for sample size determination in biospectroscopy research.