Vision-Assisted Foundation Model for Solving Multitask Vehicle Routing Problems

Multitask vehicle routing problems (VRPs) play a critical role in enhancing efficiency across various industries and service sectors. These problems consist of multiple variants that optimize routing costs while meeting diverse customer constraints. Existing multitask VRP solvers solely utilize a graph-based modality, limiting their ability to address variants with multiple constraints. As a format to represent complex semantics, the vision modality shows great potential for encoding diverse VRP constraints. This motivates us to learn patch-level semantics from the vision images, and then integrate them into a graph-based model to solve various VRP variants simultaneously. However, directly applying this approach to multitask VRPs presents three challenges: 1) existing VRP images lack constraint representations, which are essential for multitask VRPs; 2)the fixed receptive field of individual patches cannot effectively accommodate varying requirements across tasks; and 3) imbalanced pixel distribution among constraints may cause the model to overlook constraints with fewer pixels. In this article, we propose a vision-assisted foundation model (VaFM) to address these challenges. In the vision modality, input images tailored to all constraints are encoded by a convolutional neural network (CNN). The obtained patch embeddings are fused with graph-based nodes to generate solutions, with an auxiliary task designed to address the pixel-imbalanced issue. In particular, we design a hybrid cross-attention fusion module to enable adaptive receptive fields for different tasks. It incorporates feature maps from shallow layers to focus on local details and uses cross-patch attention to capture global information. Moreover, we design a constraint-aware auxiliary task and utilize binary cross-entropy (BCE) loss to ensure balanced learning of all constraints. The performance of VaFM is evaluated across 16 different VRP variants. The experimental results demonstrate the superiority of VaFM over state-of-the-art (SOTA) methods, especially for variants with complex constraints.