More is Better: Precise and Detailed Image Captioning Using Online Positive Recall and Missing Concepts Mining

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Automatic image captioning has advanced using semantic concepts.
• Existing concept-to-caption frameworks suffer from insufficient concepts due to data imbalance and incomplete labeling.

Purpose of the Study:

• To propose a method to overcome concept deficiency in image captioning.
• To improve the accuracy and detail of generated image captions.

Main Methods:

• Introduced 'online positive recall and missing concepts mining' to address concept imbalance and incomplete labeling.
• Employed adaptive loss re-weighting for online positive recall.
• Utilized a two-stage optimization for mining missing concepts.
• Implemented an element-wise concept selection process during caption generation.

Main Results:

• The proposed method successfully detects more semantic concepts.
• Achieved superior image captioning performance compared to competitive methods on the MSCOCO dataset.
• Generated more precise and detailed image descriptions.

Conclusions:

• The novel method effectively enhances semantic concept detection for image captioning.
• The approach leads to significant improvements in caption quality and accuracy.
• This work contributes to advancing the field of automatic image understanding and generation.