Artificial Intelligence in Detecting Statistical Errors: Implications for Authors, Reviewers, and Editors

Area of Science:

• Research Integrity
• Statistical Analysis
• Artificial Intelligence in Science

Background:

• Statistical errors can lead to incorrect research conclusions, compromising scientific integrity.
• Research integrity demands honest, clear presentation and correct statistical methods.
• Artificial intelligence (AI) systems are emerging as tools to detect statistical errors and aid researchers.

Purpose of the Study:

• To evaluate the role and effectiveness of AI in identifying statistical errors in research.
• To explore how AI tools can assist in maintaining research integrity and improving peer review.
• To understand the capabilities and limitations of AI in statistical analysis for scientific research.

Main Methods:

• Review of AI tools such as Statcheck, GRIM-Test, LLMs, Black Spatula, and YesNoError for statistical error detection.
• Analysis of AI performance in identifying errors in methodology, citations, and statistical analyses.
• Assessment of AI accuracy in controlled versus complex data analysis scenarios.

Main Results:

• AI tools, particularly Statcheck and GRIM-Test, show promise in spotting statistical errors, increasing research reliability.
• AI demonstrates moderate overall accuracy, with better performance in controlled settings.
• AI can expedite peer review and reduce reviewer workload but has limitations including bias and lack of expert judgment.

Conclusions:

• AI offers valuable, though imperfect, support for research integrity and statistical accuracy, especially with increasing retractions.
• Effective and safe AI implementation requires large datasets, interdisciplinary collaboration, and secure systems.
• Human oversight is indispensable for final decision-making, ensuring responsible AI utilization in research.