Comparing machine learning and artificial neural network models in psychological research: a ROC-based analysis

The increasing use of data-driven methods in psychological assessment has raised the question of whether artificial neural networks provide advantages over established machine learning approaches in applied selection contexts. In particular, comparative evidence based on ROC-based evaluation using real-world psychological datasets remains limited.

Using a dataset of N = 4,155 applicants from a university entrance examination, this study compared three traditional machine learning models-logistic regression, decision tree, and random forest-with a feedforward artificial neural network comprising a single hidden layer. All models were implemented in Python and evaluated using accuracy and receiver operating characteristic (ROC) analysis, with the area under the curve (AUC) as the primary performance metric.

Logistic regression achieved the highest predictive performance (accuracy = 0.973, AUC = 0.99), followed closely by the random forest model (accuracy = 0.961, AUC = 0.98). The artificial neural network reached competitive accuracy (0.933) but showed reduced discriminative ability (AUC = 0.87) and indications of overfitting. Feature importance analyses consistently identified biology, chemistry, and numerical reasoning as the most influential predictors of admission success.

The results indicate that for medium-sized, structured psychological datasets, traditional machine learning models provide more stable, interpretable, and robust performance than the evaluated shallow neural network architecture. These findings highlight the importance of model choice and inductive bias in applied psychological research and support the continued use of classical machine learning approaches in selection and assessment contexts.