Intra- and Inter-Head Orthogonal Attention for Image Captioning.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Multi-head attention (MA) is crucial for image captioning, enabling models to focus on key information from different representation subspaces.
• Current MA methods lack mechanisms to ensure appropriate attention distribution across subspaces, leading to over-focused heads and redundancy.
• This limits the diversity and representation power of attention mechanisms in image captioning.

Purpose of the Study:

• To propose a novel Intra- and Inter-Head Orthogonal Attention (I²OA) mechanism to enhance MA for image captioning.
• To address the issues of over-focused attention and head redundancy in existing MA models.
• To improve the performance of image captioning models without increasing complexity or parameters.

Main Methods:

• Introduced Intra-Head Orthogonal Attention to decentralize attention from object-centric to content-aware by applying orthogonal constraints within each head.
• Implemented Inter-Head Orthogonal Attention to reduce redundancy between heads by applying orthogonal constraints across heads, enhancing subspace diversity.
• Integrated I²OA into existing multi-head attention-based image captioning frameworks.

Main Results:

• The proposed I²OA method effectively improved the performance of image captioning models on the MS COCO dataset.
• Intra-Head Orthogonal Attention led to more comprehensive content-aware attention.
• Inter-Head Orthogonal Attention successfully reduced head redundancy and increased representation diversity.

Conclusions:

• I²OA offers an efficient and flexible approach to improve multi-head attention for image captioning.
• The orthogonal regularization effectively addresses limitations in attention focus and head redundancy.
• The method demonstrates significant performance gains without additional model complexity or parameters.