Comparing Genomic Prediction Models by Means of Cross Validation

Area of Science:

• Quantitative genetics
• Statistical genomics
• Plant breeding

Background:

• Genomic selection (GS) has seen two decades of development, with numerous models proposed for prediction using dense marker panels.
• Model selection and hyper-parameter optimization are crucial for maximizing predictive accuracy in genomic selection.

Purpose of the Study:

• To discuss and illustrate the use of cross-validation for optimizing hyper-parameters and comparing genomic selection models.
• To introduce statistical methods for assessing relevant differences in model performance, inspired by clinical research.

Main Methods:

• Utilized publicly available crop datasets for illustration.
• Employed paired comparisons and cross-validation, specifically paired k-fold cross-validation.
• Introduced equivalence margins and new statistical tests for performance assessment.

Main Results:

• Most hyper-parameters can be effectively learned by minimizing Restricted Maximum Likelihood (REML) or using weakly-informative priors.
• Default software options often yield competitive results compared to optimized values.
• Paired k-fold cross-validation is a statistically powerful and generally applicable method for assessing differences in model accuracies.

Conclusions:

• Cross-validation, particularly paired k-fold, is a robust tool for genomic selection model assessment.
• Defining equivalence margins based on expected genetic gain enhances the utility of these methods for breeders.
• Optimizing hyper-parameters and selecting models using these validated approaches can improve predictive performance in breeding programs.