A localization strategy combined with transfer learning for image annotation.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Automatic image annotation faces challenges with overfitting due to limited labeled data.
• Existing methods struggle to accurately assign labels to images, especially in multilabel scenarios.

Purpose of the Study:

• To address the overfitting problem in automatic image annotation.
• To propose a novel transfer learning model, CNN-2L, incorporating a label localization strategy.
• To improve the performance of multilabel image annotation.

Main Methods:

• Utilized InceptionV3 network pretrained on ImageNet for feature extraction.
• Integrated a Squeeze and Excitation (SE) module for feature reweighting.
• Implemented a label localization algorithm to derive label probabilities and determine optimal label sets.
• Experimentally determined the optimal number of labels (K value) to solve the empty label set problem.

Main Results:

• CNN-2L demonstrated significant improvements in labeling precision and recall on the Corel5k dataset.
• Achieved 18% and 15% higher precision compared to MBRM and JEC algorithms, respectively.
• Improved recall by 6% over JEC and outperformed Weight-KNN and AHL in precision.
• Showcased a 1% improvement in F1 value compared to the Semantic Extension Model (SEM).

Conclusions:

• The proposed CNN-2L model effectively enhances automatic image annotation performance.
• The label localization strategy is crucial for boosting multilabel image annotation accuracy.
• CNN-2L offers a robust solution for scenarios with insufficient labeled image data.