Mm-VitnNet: a gated image-text interaction network for soybean salt tolerance recognition using chlorophyll...

Area of Science:

• Agricultural Science
• Plant Physiology
• Artificial Intelligence in Agriculture

Background:

• Traditional salt tolerance identification in soybeans is laborious and time-consuming.
• Chlorophyll fluorescence imaging offers potential but faces challenges in data analysis and utilization.
• Limited exploration of multimodal data (images and text) hinders accurate phenotyping.

Purpose of the Study:

• To develop an efficient and accurate method for identifying soybean salt tolerance levels.
• To leverage multimodal data, including chlorophyll fluorescence images and associated text data.
• To propose a novel deep learning model for enhanced cross-modal interaction and feature learning.

Main Methods:

• Conducted salt stress experiments on 178 soybean varieties.
• Constructed a multimodal dataset using chlorophyll fluorescence imaging.
• Developed a novel gated mechanism network for learnable image-text interaction (Mm-VitnNet) enabling global cross-modal interaction.

Main Results:

• The Mm-VitnNet achieved a high accuracy rate of 98.97% in identifying salt tolerance levels.
• Demonstrated superior performance compared to existing CNN, Transformer, and hybrid models.
• Achieved an effective balance between accuracy and computational efficiency (10.22M parameters, 1.84G FLOPs).

Conclusions:

• The Mm-VitnNet provides a highly accurate and efficient non-destructive method for assessing soybean salt tolerance.
• This multimodal approach enhances agricultural phenotyping precision and intelligence.
• The model performs reliably even with limited computational resources, offering a feasible solution for practical applications.