Evaluation of a decided sample size in machine learning applications.

Area of Science:

• Machine learning
• Statistical analysis
• Research methodology

Background:

• Appropriate sample size is critical for reliable study outcomes in machine learning (ML).
• Inadequate samples lead to overfitting and reduced probability of true effects.
• Current sample size methods may be biased due to miscalculations or missing experimental details.

Purpose of the Study:

• To design criteria for evaluating sample size adequacy in ML studies.
• To examine the impact of sample size on ML classifier performance and effect sizes.
• To derive practical guidelines for sample size selection in ML research.

Main Methods:

• Systematically increased sample size from 16 using random sampling on simulated and real datasets.
• Evaluated the performance of five ML methods and calculated average and grand effect sizes.
• Utilized tenfold cross-validation to quantify classification accuracy.

Main Results:

• Effect sizes and classification accuracies increased with sample size in datasets with good discriminative power.
• Indeterminate datasets showed poor effect sizes and accuracies, unaffected by sample size increases.
• Derived criteria: suitable sample size requires effect size ≥0.5 and ML accuracy ≥80%.

Conclusions:

• Proposed practical criteria for evaluating sample size adequacy in ML studies.
• These criteria can serve as a reference for authors and editors.
• Ensures a good cost-benefit ratio by identifying optimal sample sizes.