Image Based Mango Fruit Detection, Localisation and Yield Estimation Using Multiple View Geometry

Area of Science:

• Agricultural Engineering
• Computer Vision
• Robotics

Background:

• Accurate fruit yield estimation is crucial for commercial orchards.
• Traditional methods require labor-intensive calibration and are prone to occlusion issues.
• Automated solutions are needed for efficient and precise yield assessment.

Purpose of the Study:

• To develop and validate a novel multi-sensor framework for identifying, tracking, localizing, and mapping individual fruits in a mango orchard.
• To overcome occlusion challenges and eliminate the need for manual field calibration in yield estimation.
• To enable precise spatial statistics and yield assessment at tree, row, and orchard block levels.

Main Methods:

• Utilized a state-of-the-art Faster R-CNN detector for fruit detection in images.
• Employed a multiple viewpoint approach with trajectory data for establishing pair-wise correspondences and solving occlusion.
• Integrated a LiDAR component for automatic canopy masking and associating fruits with specific trees.
• Applied 3D triangulation to accurately locate fruits and derive spatial statistics.

Main Results:

• Successfully scanned 522 trees and 71,609 mangoes in a commercial orchard.
• Validated the system against manual counts with an overall error rate of 1.36% for individual tree yield estimation.
• Demonstrated that the multi-view approach provides precise yield estimates without requiring calibration, unlike single or dual-view methods.

Conclusions:

• The proposed multi-sensor framework offers an efficient and accurate solution for automated mango yield estimation.
• The system's ability to perform yield estimation without calibration significantly reduces labor and improves precision.
• This technology has the potential to revolutionize orchard management and precision agriculture practices.