Structured Multimodal Deep Learning improves Genomic Prediction in Future Environments

Predicting phenotypes from genetics and environmental inputs is a long-standing challenge in genetics and plant breeding. Deep neural networks are a promising approach due to their capacity to approximate nonlinear biological processes. Despite initial expectations, recent studies have found that deep neural networks under-perform compared to linear methods, even on continent-scale datasets. We attribute this to several failure modes of deep learning, including greedy learning, the tendency to over-emphasize a single type of input data. As a solution, we present the Structured Interaction Neural Network (SINN), which combines statistical decomposition of genetic, environmental and interaction effects with deep neural networks. SINN dissects phenotype prediction into isolated component modeling tasks, revealing poor generalization of learned representations to new environments as the main limitation for prediction of genotype-by-environment interactions and overall yield. We reach competitive performance on yield prediction in the next cycle of 2 maize multi-environment trial datasets, including new genotypes and environments. In the Genomes to Fields (United States) dataset, SINN achieved higher overall accuracy (0.63) than BLUP-based methods (0.43) and a neural network from previous literature (0.48), and surpassed previous top-performing models with a lower RMSE (2.40 vs. 2.46 Mg/ha, mean yield 9.51 Mg/ha). Similar gains were observed in a secondary dataset consisting of Chinese national maize variety trials (MaizeGEP). SINN achieved higher overall accuracy (0.79) than BLUP-based methods (0.49) and a neural network from previous literature (0.76). By combining statistical genetics and modern deep learning, SINN enables accurate, modular and scalable genomic prediction in new environments.