Multi-modal deep learning for paddy health assessment: fusing leaf imagery with tabular metadata using a factorized...

Area of Science:

• Agricultural Science
• Computer Science
• Artificial Intelligence

Background:

• Accurate crop disease diagnosis is crucial for global food security, with paddy rice being a vital staple.
• Traditional methods are labor-intensive and require domain expertise, hindering efficient farm management.
• Existing deep learning models often miss contextual information vital for disease identification.

Purpose of the Study:

• To develop a novel multi-modal deep learning framework for precise paddy rice health assessment.
• To integrate visual leaf image data with tabular information (paddy type, days) for holistic analysis.
• To overcome limitations of conventional methods and simple feature concatenation in disease diagnosis.

Main Methods:

• Proposed a Multi-Modal Factorized Bilinear Pooling (MFBP) model.
• Combined high-level visual features from leaf images with tabular data.
• Utilized Factorized Bilinear Pooling (FBP) to capture complex interactions between diverse data types.
• Tested on the Paddy Doctor: Paddy Disease Classification dataset (>10,000 images).

Main Results:

• The MFBP model demonstrated superior performance compared to baseline concatenation fusion.
• Effectively encoded complex interactions between visual and tabular features for subtle pattern recognition.
• Showcased the ability to learn context-specific disease indicators (e.g., blemish, species, age).

Conclusions:

• Multi-modal deep learning offers a more robust and accurate approach to crop disease diagnosis.
• The MFBP model provides a precise and context-aware tool for paddy health management.
• Highlights potential for AI in advancing precision agriculture and sustainable farming practices.