Visual-textual joint relevance learning for tag-based social image search.

Area of Science:

• Information Science
• Computer Science
• Artificial Intelligence

Background:

• Social media image search is popular, driving research into tag-based retrieval.
• Existing methods often process visual and textual information separately or sequentially.
• There's a need for integrated approaches to improve social image search relevance.

Purpose of the Study:

• To propose a novel approach for estimating the relevance of user-tagged images.
• To simultaneously utilize both visual and textual information for improved search accuracy.
• To leverage hypergraph learning for integrated feature analysis in social image retrieval.

Main Methods:

• Developed a hypergraph learning framework for social image relevance estimation.
• Constructed a social image hypergraph with images as vertices and visual/textual terms as hyperedges.
• Employed pseudo-positive images to update hyperedge weights during learning, enabling automatic modulation of term importance.

Main Results:

• The proposed method effectively integrates visual and textual information for image relevance.
• Experimental results on a dataset of over 370 images demonstrate significant improvements.
• The hypergraph learning approach automatically balances the impact of visual and textual features.

Conclusions:

• Simultaneous utilization of visual and textual data via hypergraph learning enhances social image search.
• The proposed method offers a more effective way to estimate image relevance compared to sequential or separate approaches.
• This research contributes to advancing tag-based social image retrieval systems.