PME: pruning-based multi-size embedding for recommender systems

Area of Science:

• Machine Learning
• Artificial Intelligence
• Data Science

Background:

• Traditional embedding techniques in recommendation models use fixed sizes for all features, leading to potential memory inefficiencies.
• Existing methods for customizing embedding sizes often result in performance degradation or high computational costs.

Purpose of the Study:

• To develop an efficient framework for allocating customized embedding sizes in recommendation models.
• To reduce memory usage and the number of parameters in embedding layers without compromising model performance.

Main Methods:

• Proposes a Pruning-based Multi-size Embedding (PME) framework that approaches size allocation from a pruning perspective.
• Prunes embedding dimensions with minimal impact on model performance during a search phase.
• Transfers the capacity of pruned embeddings to obtain customized token sizes with reduced search cost.

Main Results:

• PME framework efficiently identifies appropriate embedding sizes for categorical features.
• Achieves strong recommendation model performance while significantly reducing the number of parameters.
• Demonstrates substantial memory savings compared to traditional fixed-size embedding methods.

Conclusions:

• The PME framework offers an effective and computationally efficient solution for optimizing embedding sizes in recommendation systems.
• Enables significant parameter reduction and memory savings, making recommendation models more scalable.