Contextual image caption creation using object positional embedding and generative models.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Automated image captioning is challenging due to limitations in capturing semantic relationships.
• Traditional deep learning models often produce captions lacking contextual richness.
• Lexical overlap is insufficient for generating semantically accurate image descriptions.

Purpose of the Study:

• To develop an advanced encoder-decoder framework for generating context-aware image captions.
• To integrate YOLOv5 object detection with a generative transformer for enhanced captioning.
• To evaluate the proposed model against established baseline methods.

Main Methods:

• An encoder-decoder framework combining YOLOv5 and a generative transformer was proposed.
• The model was benchmarked against CNN-LSTM (M1) and a BERT-based transformer (M2).
• Evaluation metrics included BLEU, ROUGE-L, METEOR, SPICE, and CIDEr.
• Expert-based evaluation assessed semantic accuracy, visual grounding, and caption usefulness.

Main Results:

• The proposed model achieved the highest CIDEr (1.10) and SPICE (0.25) scores, indicating superior semantic understanding.
• Expert evaluations showed 93% accuracy in semantic alignment with human interpretation.
• While BLEU scores were comparable to baselines, other metrics demonstrated significant improvements in caption quality and contextual relevance.

Conclusions:

• The proposed YOLOv5 and generative transformer framework significantly enhances automated image captioning.
• The model demonstrates superior ability in capturing object relationships and semantic meaning.
• Expert evaluations confirm the model's effectiveness in generating accurate and useful image descriptions.