Multimodal learning on RGB-D image for precise litchi phenotyping and weight estimation.

Area of Science:

• Agricultural Science
• Computer Vision
• Plant Breeding

Background:

• Accurate litchi phenotyping is crucial for cultivar selection and breeding research.
• Manual measurements are time-consuming, subjective, and inefficient.
• Litchi's complex fruit surface and seed morphology pose challenges for automated analysis.

Purpose of the Study:

• To develop an automated, precise phenotyping tool for litchi fruit traits.
• To address the difficulties in semantic segmentation and estimation caused by litchi's unique characteristics.
• To improve the efficiency and objectivity of litchi breeding programs.

Main Methods:

• Developed LitchiPhenoNet, a multimodal deep learning framework using RGB and depth data.
• Employed a dual-branch architecture with an RD-Fusion module for cross-modal feature extraction.
• Utilized an RGB-D dataset of 1,198 litchi image pairs across 10 cultivars.

Main Results:

• LitchiPhenoNet achieved millimeter-level diameter estimation (R² ≈ 0.98, ME < 2mm), outperforming YOLO models.
• Gram-level precision was obtained for whole fruit, pit, and pulp weight estimation (R² up to 0.98).
• The framework demonstrated robustness against complex pericarp surfaces and cross-cultivar variability.

Conclusions:

• LitchiPhenoNet offers an efficient, reliable, and accurate solution for litchi phenotypic trait quantification.
• The framework enhances objectivity and efficiency in litchi breeding and selection.
• The approach is extensible to other textured fruits and scalable for high-throughput phenotyping.