Enhancing image caption generation through context-aware attention mechanism.

Area of Science:

• Artificial Intelligence
• Computer Vision
• Natural Language Processing

Background:

• Image captioning research primarily focuses on high-resource languages like English.
• Low-resource languages, such as Bengali, face unique challenges in generating coherent image descriptions.
• Accurate object diagnosis and contextual understanding are crucial for effective image captioning.

Purpose of the Study:

• To propose a context-aware attention mechanism for improved image captioning in Bengali.
• To address the challenges of low-resource language captioning and domain knowledge transfer.
• To enhance the accuracy of linking visual objects with corresponding Bengali words.

Main Methods:

• Utilized ResNet-50 for image feature encoding to address vanishing gradients and complex features.
• Implemented a bidirectional Gated Recurrent Unit (GRU) with an attention mechanism for caption decoding.
• Developed a context-aware attention mechanism over semantic attention for precise object diagnosis.

Main Results:

• Achieved significant performance improvements on three Bengali datasets (BAN-Cap, BanglaLekhaImageCaption, Bornon).
• Demonstrated METEOR score increases of approximately 30%, 18%, and 45% over existing methods.
• Showcased superior performance compared to state-of-the-art models in Bengali image captioning.

Conclusions:

• The proposed context-aware, attention-based system significantly enhances Bengali image captioning.
• The model effectively captures contextual dependencies for more accurate descriptions.
• This research contributes to advancing AI capabilities in low-resource language applications.