DistMLLM: Enhancing Multimodal Large Language Model Serving in Heterogeneous Edge Computing.

Area of Science:

• Artificial Intelligence
• Edge Computing
• Machine Learning

Background:

• Multimodal Large Language Models (MLLMs) provide advanced capabilities for processing diverse data types like text, images, and audio.
• Edge deployment of MLLM services offers reduced latency but faces challenges from high computational needs and device variability.

Purpose of the Study:

• To introduce DistMLLM, a novel profit-driven framework for efficient MLLM service deployment in heterogeneous edge computing environments.
• To address the complexities of task allocation and scheduling for MLLMs at the edge, considering device capabilities and competing interests.

Main Methods:

• DistMLLM disaggregates MLLM tasks into encoding and inference stages, dynamically assigning them to edge devices based on their computational capacity.
• A multi-agent bandit algorithm is utilized to optimize task allocation and scheduling, learning to manage uncertain device conditions and provider competition.

Main Results:

• Simulations show DistMLLM significantly outperforms existing baselines in achieving higher long-term profit.
• The framework demonstrates a notable reduction in regret, indicating efficient resource utilization and adaptive decision-making.
• DistMLLM proves to be a scalable and adaptable solution for deploying MLLM services on the edge.

Conclusions:

• DistMLLM offers an effective solution for the profitable and efficient deployment of MLLM services in heterogeneous edge environments.
• The proposed framework addresses key challenges in edge AI by optimizing task allocation and resource management for demanding MLLM workloads.