Summarising and validating test accuracy results across multiple studies for use in clinical practice

Area of Science:

• Biostatistics
• Clinical Epidemiology
• Medical Decision Making

Background:

• Meta-analyses of test accuracy studies provide summary estimates but may not directly apply to new clinical populations due to heterogeneity.
• Clinicians require guidance on how to interpret meta-analysis findings for individual patients in diverse settings.
• Variations in sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) across populations necessitate refined interpretation strategies.

Purpose of the Study:

• To develop methods for predicting test accuracy (sensitivity, specificity) in new populations based on meta-analysis data.
• To provide clinicians with strategies for deriving accurate post-test probabilities (PPV, NPV) in new populations using existing meta-analysis results.
• To propose and evaluate a cross-validation approach for assessing the calibration of derived post-test probabilities.

Main Methods:

• Deriving prediction intervals and probability statements for test accuracy metrics (sensitivity, specificity) in novel populations.
• Developing strategies for clinicians to calculate post-test probabilities (PPV, NPV) informed by meta-analysis summary data.
• Employing a cross-validation framework to examine and compare the calibration performance of predicted versus observed post-test probabilities.

Main Results:

• In one example, the probability of a test having both >80% sensitivity and specificity in a new population was low (0.19) due to low sensitivity, despite a high summary PPV.
• The summary PPV (0.97) demonstrated good calibration in new populations, with a 0.78 probability that the true PPV would be at least 0.95.
• Tailoring post-test probabilities to local prevalence improved calibration, with cross-validation showing a 0.97 probability that the observed NPV would be within 10% of the predicted NPV.

Conclusions:

• Meta-analysis summary accuracy metrics may not directly translate to new populations; prediction intervals and probability statements are needed.
• Clinicians can derive more reliable post-test probabilities by tailoring predictions to specific population prevalence using meta-analysis data.
• The proposed methods and cross-validation approach enhance the clinical utility of diagnostic and prognostic test meta-analyses by addressing heterogeneity and improving calibration.