Cross-modal semantic autoencoder with embedding consensus.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Data Science

Background:

• Traditional single-modal retrieval methods struggle with heterogeneous multi-data, often neglecting semantic similarities.
• Existing approaches lack consideration for semantic relationships between different data modalities.
• The need for effective methods to bridge information gaps in diverse datasets is critical.

Purpose of the Study:

• To propose a novel cross-modal semantic autoencoder with embedding consensus (CSAEC) for improved cross-modal retrieval.
• To map heterogeneous data into a shared low-dimensional semantic space, preserving essential information.
• To enhance the accuracy and effectiveness of retrieving information across different data modalities.

Main Methods:

• Developed a cross-modal semantic autoencoder with embedding consensus (CSAEC).
• Utilized an autoencoder to associate feature projection with semantic code vectors, considering inter-modal similarity.
• Applied regularization and sparse constraints to low-dimensional matrices to balance reconstruction errors and achieve denoising.
• Transformed high-dimensional data into a semantic code vector.

Main Results:

• Experiments on four multi-modal datasets demonstrated significant improvements in query results.
• Achieved effective cross-modal retrieval, surpassing limitations of traditional methods.
• The CSAEC model showed enhanced accuracy and superior results in recognition tasks.

Conclusions:

• CSAEC offers an innovative deep learning solution for converting multi-modal data into abstract expressions.
• The model effectively addresses challenges in traditional cross-modal retrieval by focusing on semantic similarity.
• CSAEC shows potential for application in computer and network fields, including deep and subspace learning.