A Vision Language-Based Framework for Detecting Industrial Mechanical, Electrical, and Plumbing Assets Using...

Area of Science:

• Computer Vision
• Artificial Intelligence
• Industrial Asset Management

Background:

• Object detection models struggle with limited training data diversity and generalizing to new asset categories in industrial settings.
• Mechanical, electrical, and plumbing (MEP) assets present unique spatial and geometric complexities challenging current detection methods.

Purpose of the Study:

• To develop an effective approach for detecting unseen MEP assets in industrial environments.
• To leverage pre-trained vision language models and close-set object detectors for open-vocabulary asset detection using unlabeled data.

Main Methods:

• Utilized pre-trained vision language models and close-set object detectors.
• Employed Grounding DINO with Swin B transformer for initial asset detection.
• Combined Grounding DINO (Swin B) with YOLOv8 for enhanced MEP asset detection.

Main Results:

• Grounding DINO (Swin B) achieved high performance in open-vocabulary MEP asset detection (e.g., mIoU of 0.6586 for valves).
• The Grounding DINO (Swin B) and YOLOv8 combination demonstrated superior results, reaching mAP50 of 0.928 for valves and 0.778 for pumps.
• Performance was validated against fine-tuned models and fully supervised detectors.

Conclusions:

• The proposed method effectively addresses the limitations of existing object detection approaches for complex industrial assets.
• Leveraging vision language models and ensemble techniques offers a promising direction for open-vocabulary detection in challenging environments.