Parameter-efficient prompt tuning based on adaptive fused pre-trained models for 3D scene understanding

Prompt tuning in point cloud pre-trained models has achieved significant reductions in trainable parameters while maintaining promising performance, showing remarkable potential in 3D vision tasks. However, the existing methods of generating prompts from a single pre-trained model may suffer from limited expressive capability, which potentially leads to performance degradation when encountering noise, outliers, or adversarial samples. To alleviate this limitation, we propose a novel parameter-efficient prompt tuning (PEPT) framework, based on adaptive fused pre-trained models. Our framework leverages prompt information generated by diverse pre-trained models to capture various point cloud features, enhancing the model's ability to learn more comprehensive representations. Specifically, we introduce a multi-level prompt generation (MLPG) module for pre-trained point cloud models, which can efficiently aggregate features at different levels. By utilizing richer and more diverse feature representations, it generates more discriminative prompts while significantly reducing the trainable parameters of the module. Furthermore, we designed a lightweight task head (LightHead). The features extracted from each pre-trained model are adaptively fused and fed into the LightHead. By jointly optimizing different models, the robustness of the framework is improved. Extensive experiments demonstrate that our PEPT framework achieves competitive performance while using only 2% of the trainable parameters compared to full fine-tuning on most tasks.