PestScope: Exclusion-Aware Large Multimodal Model for Fine-Grained Agricultural Pest Segmentation

Area of Science:

• Computer Vision
• Agricultural Technology
• Artificial Intelligence

Background:

• Existing reasoning segmentation (RS) models lack domain-specific knowledge for agriculture, struggling with similar pest appearances and small target scales.
• Precise segmentation of agricultural pests is crucial for accurate area measurement and density analysis, tasks not well-addressed by current detection methods.

Purpose of the Study:

• To introduce a fine-grained pest RS task, including Pest Discriminative Referring Expression Segmentation (PDRES) and Pest Exclusion Reasoning Segmentation (PERS).
• To propose PestScope, a novel model integrating vision, language, and reasoning for enhanced fine-grained pest segmentation in agriculture.
• To address the scarcity of fine-grained, difficulty-controllable pest RS datasets through an automated dataset construction pipeline.

Main Methods:

• Developed PestScope, incorporating dedicated [NON] and [SEG] tokens to prioritize small target pests and suppress non-target regions.
• Introduced an Exclusivity Suppression Loss to differentiate supervision for [SEG] and [NON] tokens, improving separation of target and non-target pests.
• Created an automated dataset construction pipeline generating 45k (PDRES) and 27.6k (PERS) image-text-mask samples across 18 pest categories.

Main Results:

• PestScope integration improved average gIoU by 4.28% (PDRES) and 6.49% (PERS) in small and similar pest scenarios.
• Demonstrated strong generalization capabilities, with gIoU increases of 21.72% (PDRES) and 8.66% (PERS) for unseen pest categories.
• The automated dataset construction pipeline successfully generated a large-scale, fine-grained pest RS dataset.

Conclusions:

• PestScope effectively addresses the challenges of fine-grained pest segmentation in agriculture, particularly for small and similar pest types.
• The proposed methods and dataset significantly advance the capabilities of reasoning segmentation in agricultural applications.
• The model's strong performance on unseen pest categories highlights its potential for real-world agricultural pest monitoring and management.