Related Experiment Video
Updated: Sep 16, 2025
Author Spotlight: 3D Movement Assessment of Maxillary Posterior Teeth in Clear Aligner Treatment
Published on: February 23, 2024
Spin and Hacking in Machine Learning Prediction Model Studies in Dentistry: A Systematic Review
Liandi Cheng1, Yixuan Pan1, Po-Kam Wo1
1State Key Laboratory of Oral Diseases & National Center for Stomatology & National Clinical Research Center for Oral Diseases, West China Hospital of Stomatology, Sichuan University, Chengdu, Sichuan, China.
Objectives:
As machine learning and prediction model research grows in popularity, researchers may be tempted to exaggerate the value of their models through misleading reporting (spin) or statistical manipulation (hacking). This study assessed spin and potential hacking in machine learning-based prediction model studies in dentistry.
Methods:
Six databases were searched to identify studies published up to October 12, 2024. 2780 eligible studies were identified, and 1206 AUC values were extracted from abstracts for hacking analysis. For spin assessment, 209 studies were selected and evaluated based on the SPIN-PM framework.
Results:
The histogram of AUC values showed fluctuations near thresholds (0.7 and 0.8), suggesting potential AUC-hacking evidence. Spin practices were identified in 37.3% (n = 78) assessed studies, mainly through unjustified use of optimistic or positive words to describe model performance and claims of clinical applicability without external validation. Facilitators of spin were found in 39.2% (n = 82) of studies, with the most frequent being the reporting of performance measures without confidence intervals.
Conclusions:
The spin practices and facilitators were prevalent, and some evidence of hacking was found. We suggest considering the 'TRIAL' (Transparency, Reporting, Integrity, Adjustment, and Learning) principles to guide machine learning prediction model studies in dentistry, thereby minimizing spin and hacking.
