Multimodal representation learning for tourism recommendation with two-tower architecture.

Area of Science:

• Computer Science
• Artificial Intelligence
• Information Retrieval

Background:

• Personalized recommendation systems are crucial for online services, especially in tourism.
• Tourist attractions offer rich multimodal data (text, images, videos) that can enhance recommendations.
• Previous methods often struggle with the performance degradation caused by excessive feature introduction.

Purpose of the Study:

• To propose a novel heterogeneous multimodal representation learning method for tourism recommendation.
• To effectively leverage multimodal features (text and image) for improved tourism recommendations.
• To address the performance limitations of existing recommendation systems when handling rich feature information.

Main Methods:

• A two-tower architecture model was developed for tourism recommendation.
• Bidirectional Long Short-Term Memory (Bi-LSTM) extracted text features.
• External Attention Transformer (EANet) extracted image features.
• Multimodal features were fused using a deep fully connected stack layer.

Main Results:

• The proposed model demonstrated superior performance compared to baseline models.
• Improvements were observed in key recommendation metrics such as Normalized Discounted Cumulative Gain (NDCG) and precision.
• The model effectively enriched item feature representation by integrating multimodal data and item IDs.

Conclusions:

• The developed multimodal representation learning method significantly enhances tourism recommendation systems.
• The fusion of text and image features using Bi-LSTM and EANet, combined with a deep fusion layer, proves effective.
• This approach offers a promising direction for improving the accuracy and expressiveness of recommender systems in the tourism domain.