Leveraging multimodal large language model for multimodal sequential recommendation

Area of Science:

• Artificial Intelligence
• Computer Science
• Machine Learning

Background:

• Conventional multimodal recommendation systems struggle with insufficient information exploitation, limited multimodal feature recognition, and ineffective dynamic preference modeling.
• Existing approaches often rely on unimodal data, failing to capture cross-modal preferences and the evolution of user interests in sequential interactions.
• Multimodal large language models (MLLMs) offer advanced cross-modal comprehension and world knowledge, presenting a promising avenue for recommendation system enhancement.

Purpose of the Study:

• To introduce MLLM-SRec, a novel sequential recommendation architecture leveraging MLLMs to address limitations in current multimodal recommendation systems.
• To develop a multimodal feature fusion mechanism for unified item representations, aligning vision and text while mitigating cross-modal differences and noise.
• To design a temporal-aware module for dynamic user preference modeling and integrate it with Chain-of-Thought prompting for effective knowledge transfer.

Main Methods:

• Developed a multimodal feature fusion mechanism using MLLMs to create unified semantic item representations, ensuring semantic alignment between visual and textual data.
• Implemented a temporal-aware user behavior comprehension module to capture the dynamic evolution of user preferences within sequential interaction data.
• Employed supervised fine-tuning combined with multistep Chain-of-Thought prompting to optimize knowledge transfer from pre-trained MLLMs to the recommendation task.

Main Results:

• The proposed MLLM-SRec architecture achieved significant improvements over state-of-the-art baselines across four benchmark datasets.
• The method substantially enhanced the precision of recommendation results.
• MLLM-SRec demonstrated superior robustness and adaptability in multimodal sequential recommendation scenarios.

Conclusions:

• MLLM-SRec effectively addresses the challenges of multimodal feature recognition and dynamic preference modeling in sequential recommendation.
• The architecture validates the significant potential of MLLMs for advancing sequential recommendation tasks by improving multimodal interaction data utilization.
• The findings offer new methodological insights for multimodal sequence recommendation research and highlight the benefits of MLLM integration.