Aggregate based light incremental sharding for efficient embedding table management for recommender systems.

Area of Science:

• Computer Science
• Machine Learning
• Artificial Intelligence

Background:

• Existing dynamic embedding table sharding methods often prioritize training over inference, leading to suboptimal performance in real-time recommendation systems.
• Challenges in recommendation inference include evolving co-occurrence patterns and strict latency requirements, which current sharding techniques do not adequately address.

Purpose of the Study:

• To develop a novel method for efficient dynamic embedding table sharding specifically optimized for recommendation inference.
• To address the limitations of existing training-centric sharding approaches by considering inference-specific challenges.

Main Methods:

• Proposes Aggregate based Light Incremental Sharding (ALIS), a method featuring aggregate based sharding for enhanced stability and quality.
• Incorporates light incremental sharding to reduce dynamic sharding costs through iterative and statistics-based approaches.

Main Results:

• ALIS demonstrates superior performance in reducing network overhead compared to state-of-the-art methods.
• The proposed method significantly improves inference efficiency in recommendation systems.

Conclusions:

• ALIS offers a rational and superior solution for dynamic embedding table sharding in recommendation inference.
• The method effectively tackles inference-specific challenges, leading to improved system performance.