Multi-Modal Multi-Scale Deep Learning for Large-Scale Image Annotation.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Large-scale image annotation involves assigning multiple labels to images.
• Challenges include learning effective feature representations and determining the optimal number of labels.

Purpose of the Study:

• To develop a method for learning rich feature representations for diverse visual concepts.
• To introduce a technique for predicting the optimal number of labels for image annotation.

Main Methods:

• A novel multi-scale deep model with a two-branch architecture (main and feature fusion) was proposed.
• The model incorporates multi-modal learning using noisy user tags alongside image data.
• An auxiliary task for label quantity prediction was integrated with the main label prediction task.

Main Results:

• The proposed multi-scale deep model effectively extracts rich and discriminative features.
• The label quantity prediction task accurately estimates the optimal number of labels.
• Experiments on benchmark datasets demonstrate significant performance improvement over state-of-the-art methods.

Conclusions:

• The developed approach enhances feature representation and optimizes label prediction in large-scale image annotation.
• This method offers a robust solution for accurately annotating images with diverse visual concepts.