Covariance Attention for Semantic Segmentation.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Semantic segmentation relies on contextual cues from global and local information.
• Current attention methods for this task exhibit high computational complexity.
• Pixel-wise correlation is a common but inefficient approach.

Purpose of the Study:

• Introduce a novel attention module for semantic segmentation.
• Address the high space and time complexity of existing attention methods.
• Enhance segmentation performance by effectively integrating diverse feature types.

Main Methods:

• Developed a new attention module utilizing covariance matrices to model feature map dependencies.
• Formulated local-global dependency as a matrix projection process.
• Proposed the Covariance Attention Network (CANet) framework for semantic segmentation.

Main Results:

• Covariance attention effectively captures global and local feature dependencies.
• Integration of hand-engineered and learned features via covariance matrices boosted segmentation performance.
• CANet achieved competitive results compared to state-of-the-art methods.

Conclusions:

• Covariance attention offers an efficient and effective mechanism for semantic segmentation.
• The proposed CANet framework demonstrates superior performance.
• Covariance matrices provide a powerful tool for encoding joint distribution information in deep learning models.