MMAgentRec, a personalized multi-modal recommendation agent with large language model.

Area of Science:

• Artificial Intelligence
• Computer Science
• Information Technology

Background:

• Multimodal recommendation systems face challenges with complex user requirements and data scarcity.
• Existing systems struggle with fragmented information and unnatural user interactions.
• Developing robust datasets for evaluating large models and human-temporal interactions is crucial.

Purpose of the Study:

• To develop an intelligent multimodal recommendation system capable of autonomous decision-making and self-reflection.
• To address pain points in multimodal recommendation, including data scarcity and unclear user needs.
• To enhance user experience through more accurate and coherent suggestions.

Main Methods:

• Integration of multimodal techniques (cross-attention, multi-graph neural networks, residual networks) with a large language model (LLM).
• Implementation of self-reflection mechanisms within the LLM for improved decision-making.
• Development of a recommendation module that consults domain experts based on user requirements.

Main Results:

• The proposed multimodal system demonstrates superior performance over classic algorithms like Blip2 and Clip in understanding user intent.
• Ablation studies confirm the critical roles of the LLM, Multi-Graph Convolutional Network (MGCN), and Cross-Attention in achieving high accuracy (0.9526) and F1 scores (0.94).
• The system outperforms state-of-the-art methods such as LightGCN and DualGNN, with MGCN and Cross-Attention showing significant improvements in recall and classification tasks.

Conclusions:

• The developed multimodal recommendation system effectively overcomes limitations of existing approaches by leveraging LLMs and advanced neural network architectures.
• The system exhibits strong capabilities in understanding user intent and generating intelligent suggestions, leading to enhanced user satisfaction.
• This research offers novel perspectives and practical applications for the evolution of multimodal recommendation systems in the information technology domain.