ModuCLIP: multi-scale CLIP framework for predicting foundation pit deformation in multi-modal robotic systems.

Area of Science:

• Geotechnical Engineering
• Robotics
• Computer Vision

Background:

• Foundation pit deformation prediction is crucial for underground engineering safety.
• Traditional methods face challenges with complex geological conditions and multi-modal data integration.
• Deep learning, especially cross-modal architectures, shows promise but requires effective data fusion.

Purpose of the Study:

• To propose ModuCLIP, a Multi-Scale Contrastive Language-Image Pretraining framework for foundation pit deformation prediction.
• To enhance prediction accuracy and robustness by integrating multi-source data (images, text, sensor data).
• To improve adaptability to complex engineering conditions using multi-modal robotic systems.

Main Methods:

• Developed a Multi-Scale Contrastive Language-Image Pretraining (CLP) framework named ModuCLIP.
• Employed a self-supervised contrastive learning mechanism for multi-source information integration.
• Utilized a multi-scale feature learning approach for enhanced adaptability.

Main Results:

• ModuCLIP demonstrated superior prediction accuracy compared to existing methods.
• The framework showed improved generalization and robustness across multiple foundation pit datasets.
• Experimental results validate the effectiveness of multi-modal data integration.

Conclusions:

• ModuCLIP offers an efficient and precise solution for foundation pit deformation prediction.
• The study provides insights into multi-modal robotic perception for engineering monitoring.
• This framework advances the application of deep learning in underground engineering safety.