ESAMask: Real-Time Instance Segmentation Fused with Efficient Sparse Attention.

Area of Science:

• Computer Vision
• Deep Learning
• Image Segmentation

Background:

• Instance segmentation requires distinguishing objects and predicting dense areas, a task often dominated by complex, high-parameter models.
• Existing models achieve high accuracy but often lack practical speed for real-time applications.
• Balancing accuracy and speed is crucial for practical instance segmentation.

Purpose of the Study:

• To introduce ESAMask, a novel real-time instance segmentation model.
• To achieve a superior accuracy-speed trade-off through efficient, lightweight design.
• To enhance feature perception and reduce computational cost in segmentation.

Main Methods:

• Developed ESAMask, a real-time segmentation model incorporating efficient sparse attention.
• Introduced a dynamic and sparse Related Semantic Perceived Attention (RSPA) mechanism for adaptive feature extraction.
• Designed the GSInvSAM structure to minimize redundant calculations and enhance feature merging.
• Integrated a Mixed Receptive Field Context Perception Module (MRFCPM) for multi-scale feature representation.

Main Results:

• ESAMask achieved a mask Average Precision (AP) of 45.4 on the COCO dataset.
• The model reached a frame rate of 45.2 FPS, demonstrating real-time performance.
• ESAMask surpassed current instance segmentation methods in the accuracy-speed trade-off.

Conclusions:

• ESAMask provides an effective solution for real-time instance segmentation with a strong accuracy-speed balance.
• The proposed attention mechanism and structural designs contribute to efficient and high-quality segmentation.
• The method demonstrates robust performance across various object classes and scales.