Related Experiment Video
Updated: Aug 7, 2026

A Machine Learning Approach to Design an Efficient Selective Screening of Mild Cognitive Impairment
Published on: January 11, 2020
Post hoc choice of cut points introduced bias to diagnostic research
1Centre for Clinical Epidemiology, University of Newcastle, Maddison Building , Level 3, NSW, Australia. Ben.Ewald@newcastle.edu.au [corrected]
Background And Objective:
To examine the extent of bias introduced to diagnostic test validity research by the use of post hoc data driven analysis to generate an optimal diagnostic cut point for each data set.
Methods:
Analysis of simulated data sets of test results for diseased and nondiseased subjects, comparing data driven to prespecified cut points for various sample sizes and disease prevalence levels.
Results:
In studies of 100 subjects with 50% prevalence a positive bias of five percentage points of sensitivity or specificity was found in 6 of 20 simulations. For studies of 250 subjects with 10% prevalence a positive bias of 5% was observed in 4 of 20 simulations.
Conclusion:
The use of data-driven cut points exaggerates test performance in many simulated data sets, and this bias probably affects many published diagnostic validity studies. Prespecified cut points, when available, would improve the validity of diagnostic test research in studies with less than 50 cases of disease.
Related Concept Videos
Bias in Epidemiological Studies
Bias
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
Decision Making: Traditional Method
First, a specific claim about the population parameter is decided based on the research question and is stated in a simple form. Further, an opposing statement to this claim is also stated. These statements can act as null and alternative hypotheses, out of which a null hypothesis would be a...
Receiver Operating Characteristic Plot
Detection of Gross Error: The Q Test
Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast, controlled...