Defining 'optimal' test cut-off using global test metrics: evidence from a cognitive screening instrument

Area of Science:

• Neurology
• Psychometrics

Background:

• Accurate diagnosis of dementia relies on effective cognitive screening instruments.
• Test cut-off selection significantly influences diagnostic accuracy metrics.

Purpose of the Study:

• To investigate how global test accuracy metrics vary with different test cut-off points for dementia diagnosis.
• To compare metrics derived from receiver operating characteristic (ROC) curves with those independent of ROC curves.

Main Methods:

• Utilized data from a test accuracy study of the Mini-Addenbrooke's Cognitive Examination (MACE).
• Calculated and plotted global accuracy measures against various test cut-off values.
• Included metrics such as Youden index and correct classification accuracy.

Main Results:

• Identified distinct 'optimal' cut-points for different global measures, showing a ten-point spread on the MACE scale.
• Observed substantial variation in test sensitivity based on the chosen optimum.
• All tested cut-offs demonstrated high negative predictive value.

Conclusions:

• The methodology for determining cut-off points in cognitive screening tools has profound implications for their performance.
• Different cut-off selection strategies can lead to divergent conclusions about a test's diagnostic utility.