Small LLMs can be good coldstart recommenders.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Natural Language Processing

Background:

• Large Language Models (LLMs) have significantly advanced AI since 2022.
• State-of-the-art LLMs (e.g., Llama 3.1, PaLM) require substantial computational resources, limiting accessibility.
• Recent research focuses on developing smaller, efficient LLMs and applying them to domain-specific tasks like recommendation systems.

Purpose of the Study:

• To investigate the efficacy of fine-tuning small LLMs (≤2 billion parameters) for sequential recommendation.
• To compare the performance of fine-tuned small LLMs against established sequential recommendation models.
• To evaluate performance, particularly in challenging cold-start scenarios.

Main Methods:

• Fine-tuning two small Large Language Models (LLMs) with 2 billion parameters or less.
• Utilizing prompt engineering and fine-tuning techniques for domain-specific application.
• Evaluating performance on sequential recommendation tasks, including cold-start settings.

Main Results:

• Fine-tuned small LLMs achieve performance comparable to larger models.
• Small LLMs demonstrate competitive or superior performance compared to baseline models like GRU4Rec and SASRec.
• Enhanced effectiveness observed in cold-start recommendation scenarios.

Conclusions:

• Small, fine-tuned LLMs are a viable and efficient alternative for sequential recommendation tasks.
• These models offer a practical solution for deploying advanced recommendation systems without requiring massive computational infrastructure.
• The approach shows promise for improving recommendation performance, especially when user data is sparse.