Image Recognition by Predicted User Click Feature With Multidomain Multitask Transfer Deep Network.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Click features, derived from user click data, effectively reduce the semantic gap in image recognition.
• Traditional image datasets lack click data, necessitating click prediction models trained on auxiliary datasets.
• Existing methods face limitations due to independent model training and cross-domain transfer challenges.

Purpose of the Study:

• To develop a unified framework for simultaneous image recognition and click prediction.
• To overcome limitations of independent click prediction models and cross-domain transfer issues.
• To enhance the effectiveness of click features in image recognition tasks.

Main Methods:

• A multitask and multidomain deep network with varied modals (MTMDD-VM) was devised.
• Datasets with and without click information were integrated for unified training.
• Nonlinear word embedding and a position-sensitive loss function were employed to capture visual click correlations.

Main Results:

• Nonlinear word embedding and position-sensitive loss improved click feature effectiveness, boosting recognition accuracy by 32%.
• The multitask learning framework enhanced performance in both image recognition and click prediction.
• Unified training with combined datasets further improved overall performance, demonstrating scalability and one-shot learning capabilities.

Conclusions:

• The proposed MTMDD-VM framework effectively integrates image recognition and click prediction.
• The method significantly improves accuracy and addresses limitations of previous approaches.
• The approach offers superior performance, scalability, and one-shot learning ability compared to state-of-the-art methods.