Predicting retracted research: a dataset and machine learning approaches.

Area of Science:

• Bibliometrics
• Scientific publishing
• Machine learning

Background:

• Retracted scientific articles compromise research integrity and can perpetuate misinformation.
• Developing methods to identify retractions is crucial for maintaining a reliable scientific record.

Purpose of the Study:

• To create a comprehensive dataset of retracted articles with bibliographic metadata.
• To train and evaluate machine learning (ML) models for predicting article retractions.
• To assess feature importance in ML classifiers through ablation studies.

Main Methods:

• An open-access dataset was constructed by integrating Retraction Watch and OpenAlex data.
• A case-controlled design paired retracted articles with non-retracted counterparts.
• Various ML models, including traditional classifiers and language models, were trained and evaluated using accuracy, precision, recall, and F1-score.

Main Results:

• The Llama 3.2 base model demonstrated high overall accuracy.
• Random Forest achieved 0.687 precision for non-retracted articles; Llama 3.2 achieved 0.683 precision for retracted articles.
• Traditional ML classifiers generally outperformed contextual language models, with Llama 3.2 showing competitive performance.

Conclusions:

• Machine learning effectively aids in identifying retracted research, though no single model dominated all metrics.
• These findings support the development of automated tools for publishers and reviewers to detect problematic publications.
• Further research is needed to refine models and incorporate additional features for enhanced predictive accuracy.