Minimum Sample Size Requirements for Machine Learning: A Study on Diabetic Neuropathy Prediction

Early detection of diabetic neuropathy (DN) remains challenging due to its asymptomatic progression. Machine learning models hold promise for identifying patients at risk, yet most existing studies are limited by small sample sizes that hinder generalizability. In this study, we evaluated minimum sample size requirements for machine learning using a population-based dataset of 77,724 individuals with diabetes. We generated balanced subsets with varying sample sizes (n=100-25,000) and numbers of features (3-46). We trained random forest models on each configuration and evaluated performance using the receiver operating characteristic area under the curve (ROC AUC) and precision-recall area under the curve (PR AUC). Models trained on ≤500 samples showed substantial overfitting and poor generalization. At n=1,000, performance was comparable to the reference model when the feature set was restricted (3 features), but overfitting was observed as the number of features increased (≥20). Performance stabilized for sample sizes ≥3,000, without evidence of overfitting. Our findings indicate that approximately 3,000 samples are required for reliable DN prediction with random forests, and that constraining feature dimensionality is critical when working with smaller cohorts. These results provide practical guidance on data sufficiency and model design for clinical machine learning studies, particularly in data-limited settings.