Confirming SPSS Results With ChatGPT-4 and o3-mini Models

Area of Science:

• Behavioral Healthcare Data Analysis
• Artificial Intelligence in Statistics
• Computational Social Science

Background:

• Comparison of statistical software (SPSS) with AI language models (ChatGPT-4, ChatGPT-3.5-mini) for data analysis.
• Evaluation of AI's quantitative performance, interpretability, and ethical considerations in behavioral healthcare research.
• Assessment of AI's adaptability and future trends in statistical data output and interpretation.

Purpose of the Study:

• To compare the statistical output and interpretation accuracy of SPSS with ChatGPT-4 and ChatGPT-3.5-mini.
• To evaluate the methodological approaches and quantitative performance of AI models in behavioral healthcare data analysis.
• To identify limitations and areas for improvement in AI-driven statistical analysis for scientific research.

Main Methods:

• Conducted fourteen statistical analyses on two real-world behavioral healthcare datasets.
• Utilized descriptive statistics, Pearson r, Spearman's rho, t-tests, ANOVA, and regression analyses.
• Inputted verbal commands and SPSS variables into ChatGPT-4 and ChatGPT-3.5-mini for comparative analysis.

Main Results:

• High concordance between SPSS and ChatGPT-4 for fundamental statistical analyses (e.g., central tendency, simple correlations, t-tests).
• ChatGPT-4 showed minimal effect size variations compared to SPSS in simple linear regression.
• Discrepancies observed in complex analyses; ChatGPT-3.5-mini exhibited inflated correlations and erroneous results in multivariate procedures.

Conclusions:

• ChatGPT-4 demonstrates strong performance in basic statistical tests, closely mirroring SPSS.
• AI models show potential but require further validation and refinement for complex statistical analyses.
• ChatGPT-3.5-mini's inconsistencies in advanced statistical modeling limit its current dependability for scientific research applications.