YOLO-SAM an end-to-end framework for efficient real time object detection and segmentation.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Traditional YOLO methods require predefined object categories, limiting their applicability.
• YOLO-World introduced open-vocabulary detection by integrating visual language modeling and large-scale pretraining.
• Existing models face challenges in balancing detection accuracy, segmentation performance, and computational efficiency.

Purpose of the Study:

• To propose an improved object detection and segmentation model based on YOLO-World-S.
• Enhance detection efficiency and accuracy beyond the original YOLO-World baseline.
• Address limitations in computational complexity and memory usage.

Main Methods:

• Introduced large-kernel separable convolutions into the RepVL-PAN to reduce computational complexity and memory footprint.
• Incorporated a dynamic sparse attention mechanism (PSBRA module) in the Neck to lower computational cost and enable integration with EfficientSAM.
• Reconstructed the loss function to effectively manage shared features and optimization objectives between detection and segmentation tasks.

Main Results:

• Achieved a mean average precision (mAP) of 58.8% on the COCO dataset.
• Reached a speed of 308 frames per second (FPS).
• Demonstrated significant improvements in both accuracy and speed compared to the original YOLO-World-S baseline.

Conclusions:

• The proposed model effectively improves upon YOLO-World-S for open-vocabulary object detection and segmentation.
• The integration of efficient architectural modifications and a refined loss function leads to superior performance.
• This advancement offers a more practical and efficient solution for complex visual recognition tasks.