Estimating the Minimum Sample Size for Neural Network Model Fitting-A Monte Carlo Simulation Study.

Area of Science:

• Psychology
• Computer Science
• Machine Learning
• Psychometrics

Background:

• Machine learning (ML) methods, particularly neural networks (NNs), are increasingly used in psychological studies for supervised model fitting on psychometric data.
• Psychometric independent variables are often ordinal and low-dimensional, posing unique challenges for ML model performance.
• There is a lack of guidance on sample size planning for NN applications in psychology.

Purpose of the Study:

• To investigate the performance of neural networks (NNs) with varying sample sizes using simulated psychometric data.
• To determine appropriate minimum sample sizes for NN model fitting based on performance criteria.
• To evaluate the impact of ordinal independent variables on NN performance in psychological research.

Main Methods:

• A simulation study was conducted to assess NN performance across different sample sizes.
• Simulations included both linear and nonlinear relationships between variables.
• Performance criteria for minimum sample size were defined as 95% of models nearing theoretical maximum performance and 80% outperforming linear models.

Main Results:

• Neural network performance was found to be unstable when using ordinal variables as independent predictors.
• The study identified specific sample size recommendations based on the defined performance metrics.
• Results indicate potential limitations of using NNs with ordinal data common in psychological research.

Conclusions:

• Neural networks may not be suitable for psychometric data with ordinal independent variables, particularly when nonlinear relationships are present.
• Researchers should exercise caution when applying NNs to such datasets.
• Further research is needed to explore alternative ML methods or data preprocessing techniques for ordinal psychometric data.