Wheat growth parameters prediction based on dual output Bayesian neural network using multi-modal information

eaf area index (LAI) and leaf nitrogen accumulation (LNA) are key indicators of wheat growth and nitrogen nutritional status. However, existing prediction methods predominantly rely on single-modal information and single-output models, limiting their ability to characterize the complex structural and physiological traits of crops. This study aimed to develop a multimodal learning framework for the simultaneous and accurate prediction of wheat LAI and LNA.

Spectral, image, and canopy structural features were extracted from wheat canopies across different cultivars, nitrogen treatments, and growth stages. A canopy height correction-based preprocessing method was developed to improve the extraction of structural features. A Dual-Output Bayesian Neural Network (DO-BNN) was then constructed to simultaneously predict LAI and LNA. In addition, an Extreme Sample Mining (ESM) strategy and a joint loss function were introduced to strengthen the learning of complementary information across modalities and the intrinsic correlation between the two target variables.

The DO-BNN achieved its best predictive performance when all feature modalities were fused. The coefficients of determination (R²) for LAI and LNA were 0.89 and 0.77, respectively, while the corresponding relative root mean square errors (RRMSEs) were 0.15 and 0.35. Compared with single-modal and conventional single-output approaches, the proposed method provided more accurate and robust predictions of both wheat growth parameters.

The results demonstrate that integrating spectral, image, and structural information can improve the characterization of wheat canopy traits. By jointly modeling LAI and LNA, the DO-BNN effectively exploited the physiological relationship between crop growth and nitrogen accumulation. The proposed framework provides a promising approach for the high-accuracy, collaborative monitoring of wheat growth and nitrogen nutritional status.