Criteria for evaluating risk prediction of multiple outcomes

Area of Science:

• * Biostatistics
• * Bioinformatics
• * Computational Biology

Background:

• * Traditional risk prediction models focus on single outcomes, limiting their utility with complex biological data.
• * Emerging omic biomarkers generate high-dimensional data for simultaneous prediction of multiple health outcomes.
• * Evaluating multiple outcome prediction models presents challenges similar to multiplicity issues in hypothesis testing.

Purpose of the Study:

• * To define criteria for evaluating multivariate prediction models that predict multiple outcomes.
• * To introduce concepts of multivariate calibration, outcome-wise vs. individual-wise prediction (joint and panel-wise).
• * To propose definitions for sensitivity, specificity, concordance, and predictive values in a multivariate context.

Main Methods:

• * Development of definitions for multivariate calibration and different prediction senses (outcome-wise, joint, panel-wise).
• * Proposal of multivariate metrics including sensitivity, specificity, concordance, positive/negative predictive value, and relative utility.
• * Linking definitions via a multivariate probit model, summarizing accuracy using covariance with a liability vector.

Main Results:

• * Demonstrated that multivariate prediction model accuracy can be summarized by its covariance with a liability vector.
• * Illustrated the proposed concepts using a biomarker panel for early detection of eight cancers.
• * Applied the framework to polygenic risk scores for six common diseases.

Conclusions:

• * Established a framework for evaluating multivariate prediction models, crucial for omics-driven health predictions.
• * The proposed definitions and methods facilitate a more nuanced assessment of prediction accuracy for multiple outcomes.
• * This work supports the development and validation of advanced risk prediction tools in precision medicine.