A Self-Supervised Pre-Trained Transformer Model for Accurate Genomic Prediction of Swine Phenotypes

Area of Science:

• Animal Genetics
• Bioinformatics
• Machine Learning in Genomics

Background:

• Genomic prediction is vital for swine breeding, but conventional methods struggle with complex non-additive genetic effects.
• Genomic Best Linear Unbiased Prediction (GBLUP) has limitations in capturing the full spectrum of genetic variation influencing phenotypes.

Purpose of the Study:

• To develop and validate a novel deep learning framework for enhanced genomic phenotype prediction in swine.
• To improve the accuracy of predicting complex economic traits by capturing non-linear genetic signals.

Main Methods:

• A self-supervised, pre-trained encoder-only Transformer model was developed, tokenizing SNP sequences into 6-mers to learn local haplotype patterns.
• The model underwent self-supervised pre-training using a masked 6-mer prediction task on unlabeled genomic data.
• The pre-trained model was fine-tuned on labeled data for predicting phenotypic values of economic traits.

Main Results:

• The proposed Transformer model consistently outperformed GBLUP and a non-pre-trained Transformer in prediction accuracy for key economic traits.
• The self-supervised pre-training enabled the model to capture complex, non-linear genetic signals missed by linear models.
• Demonstrated superior performance in genomic prediction accuracy for swine breeding applications.

Conclusions:

• Self-supervised learning offers a powerful approach to decipher complex genomic patterns for practical applications in breeding programs.
• The novel framework provides a more accurate methodology for genomic phenotype prediction, enhancing genetic gain in swine.
• This research validates the potential of advanced machine learning techniques to accelerate genetic progress in livestock.