Probabilistic Attention Map: A Probabilistic Attention Mechanism for Convolutional Neural Networks

Area of Science:

• Computer Vision
• Machine Learning
• Deep Learning

Background:

• Attention mechanisms are crucial for Convolutional Neural Network (CNN) vision backbones in sensing and imaging.
• Conventional attention modules often rely on heuristic design and empirical tuning, presenting a significant challenge.

Purpose of the Study:

• To propose a novel probabilistic attention mechanism to address the limitations of conventional methods.
• To enhance the performance of CNNs in image classification tasks.

Main Methods:

• Estimating the probabilistic distribution of activation maps within CNNs using a Laplace distribution.
• Constructing probabilistic attention maps based on the correlation between attention weights and the estimated distribution.
• Integrating the probabilistic attention map as a plug-and-play module into existing CNN architectures via element-wise multiplication.

Main Results:

• The proposed probabilistic attention mechanism effectively boosts image classification accuracy.
• The approach demonstrates superior performance across various CNN backbone models compared to baselines and other attention mechanisms.

Conclusions:

• The novel probabilistic attention mechanism offers a principled and effective way to design attention for CNNs.
• This method provides a significant improvement in image classification accuracy and generalizability.