Comprehensive Evaluation of Multispectral Image Registration Strategies in Heterogenous Agriculture Environment

Area of Science:

• Agricultural remote sensing
• Computer vision
• Image processing

Background:

• Accurate multispectral (MS) image registration is crucial for precise agricultural monitoring and weed detection.
• Scale-Invariant Feature Transform (SIFT) and Random Sample Consensus (RANSAC) are common algorithms for image registration.
• Evaluating different registration strategies is essential for optimizing performance in complex agricultural scenes.

Purpose of the Study:

• To comprehensively evaluate three SIFT- and RANSAC-based MS image registration approaches.
• To assess registration accuracy for heterogeneous agricultural environments containing crops (Triticum aestivum) and weeds (Raphanus raphanistrum).
• To compare the performance of registration methods using manual versus automatic segmentation.

Main Methods:

• Three SIFT- and RANSAC-based registration methods were evaluated: spatial realignment of annotations, registration of binary masks from ground truth, and registration of masked pixels of interest.
• Methods were tested on MS images of Triticum aestivum and Raphanus raphanistrum.
• Performance was analyzed using both manually segmented ground truth data and automatically segmented instances (YOLOv8l-seg).

Main Results:

• Registration using binary masks from manually segmented images yielded the highest accuracy, outperforming spatial realignment and masked pixel registration.
• For automatically segmented images, registration of predicted mask instances was more accurate than masked pixel registration.
• Near-infrared and blue channels showed higher accuracy in ground truth and automatically segmented images, respectively. Instance-level accuracy varied by channel and method.

Conclusions:

• MS image registration based on binary masks, particularly from manual segmentation, offers superior accuracy for distinguishing crops and weeds.
• Automatic segmentation methods, like YOLOv8l-seg, show promise for improving registration accuracy in agricultural applications.
• Channel selection and specific registration strategies significantly impact overall accuracy, highlighting the need for tailored approaches.