Related Experiment Video
Updated: Aug 4, 2025
06:19
Constructing and Visualizing Models using Mime-based Machine-learning Framework
Published on: July 22, 2025
37
Systematic review finds "spin" practices and poor reporting standards in studies on machine learning-based prediction
Constanza L Andaur Navarro1, Johanna A A Damen1, Toshihiko Takada2
1Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands; Cochrane Netherlands, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands.
Journal of Clinical Epidemiology
|April 6, 2023
Summary
Studies developing clinical prediction models with machine learning show frequent "spin" and poor reporting. A framework is needed to improve the sound reporting of these important prediction model studies.
Area of Science:
- Clinical prediction modeling
- Machine learning in healthcare
- Biostatistics and epidemiology
Background:
- Clinical prediction models (CPMs) are crucial for patient care.
- Supervised machine learning (ML) is increasingly used to develop and validate CPMs.
- Concerns exist regarding "spin" (biased reporting) and inadequate reporting standards in ML-based CPM studies.
Approach:
- Systematic literature search of PubMed (Jan 2018-Dec 2019) for diagnostic and prognostic CPMs using supervised ML.
- Inclusion of 152 studies regardless of data source, outcome, or clinical specialty.
- Evaluation of reporting quality, focusing on precision estimates and external validation.
Key Points:
- High rates of "spin" and poor reporting were observed in ML-based CPM studies.
- Discrimination metrics were frequently reported without precision estimates (74.6% in abstracts, 65.4% in main texts).
- Many studies recommended clinical use without external validation (95.2% in abstracts, 55.6% in main texts).
Conclusions:
- "Spin" and inadequate reporting are prevalent in ML-based clinical prediction model studies.
- A dedicated framework for identifying "spin" is necessary to promote rigorous reporting.
- Improving reporting standards is essential for the reliable application of ML in clinical prediction.
Related Concept Videos
01:52Regression Toward the Mean
6.3K
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when...
6.3K
01:19Survival Tree
126
Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
Building a Survival Tree
Constructing a...
Building a Survival Tree
Constructing a...
126
01:22Bias
4.3K
Bias refers to any tendency that prevents a question from being considered unprejudiced. In research, bias occurs when one outcome or answer is selected or encouraged over others in sampling or testing. Bias can occur during any research phase, including study design, data collection, analysis, and publication.
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
4.3K
01:20Random and Systematic Errors
11.2K
Scientists always try their best to record measurements with the utmost accuracy and precision. However, sometimes errors do occur. These errors can be random or systematic. Random errors are observed due to the inconsistency or fluctuation in the measurement process, or variations in the quantity itself that is being measured. Such errors fluctuate from being greater than or less than the true value in repeated measurements. Consider a scientist measuring the length of an earthworm using a...
11.2K
01:03Prediction Intervals
2.3K
The interval estimate of any variable is known as the prediction interval. It helps decide if a point estimate is dependable.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
2.3K
02:07Improving Translational Accuracy
11.7K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
11.7K