Individual Segmentation of Intertwined Apple Trees in a Row via Prompt Engineering

Area of Science:

• Agricultural Science
• Computer Vision
• Machine Learning

Background:

• High-throughput phenotyping of horticultural crops like apple trees is crucial for breeding and variety testing.
• Accurate segmentation of individual trees in dense orchards is challenging due to complex lighting and intertwined branches.
• Traditional supervised learning methods require substantial annotated data, limiting scalability.

Purpose of the Study:

• To develop an efficient and scalable method for segmenting individual apple trees in complex orchard environments.
• To explore the use of prompt engineering with foundational models for zero-shot plant segmentation.
• To reduce the reliance on large annotated datasets for agricultural computer vision tasks.

Main Methods:

• Utilized prompt engineering with the Segment Anything Model (SAM) and its variants in a zero-shot setting.
• Implemented a YOLOv11 model for detecting apple tree trunks.
• Positioned a diamond-shaped prompt above the detected trunk to guide the SAM for tree segmentation.

Main Results:

• Achieved a 97% trunk detection rate.
• Obtained a 70% Dice score on the REFPOP dataset and 84% on a public dataset without prior training.
• Demonstrated performance comparable to or exceeding supervised methods and non-prompted foundation models.

Conclusions:

• Prompt-guided foundational models offer a scalable and annotation-efficient solution for plant segmentation in agriculture.
• This approach holds significant potential for advancing phenotyping in complex field conditions.
• The developed method effectively addresses the challenges of segmenting individual trees in dense orchard settings.