Inter-modality feature prediction through multimodal fusion for 3D shape defect detection.

Area of Science:

• Computer Vision
• Machine Learning
• Industrial Automation

Background:

• 3D shape defect detection is crucial for autonomous industrial inspection.
• Accurate anomaly detection is challenging with multimodal sensor data (RGB, depth, point clouds).
• Integrating color and structural information is often required but complex.

Purpose of the Study:

• To propose a lightweight framework for efficient 3D shape defect detection.
• To effectively utilize multimodal fused features from RGB, depth, and point clouds.
• To improve anomaly detection accuracy by leveraging complementary sensor information.

Main Methods:

• Developed a framework with modality-specific pre-trained feature extractors.
• Introduced a Multi-level Adaptive Dual-Modal Gated Fusion (ADMGF) module for RGB-Depth feature fusion.
• Implemented a lightweight inter-modal feature prediction network for bidirectional learning across modalities.
• Avoided large memory banks and pixel-level reconstructions.

Main Results:

• Achieved significant performance improvements on MVTec3D-AD and Eyecandies datasets.
• Demonstrated superior accuracy in 3D shape defect detection compared to state-of-the-art methods.
• Validated the effectiveness of the lightweight, fused-feature approach.

Conclusions:

• The proposed framework offers an efficient and accurate solution for 3D shape defect detection.
• Effective fusion of multimodal data enhances anomaly detection capabilities.
• The bidirectional learning mechanism contributes to robust defect identification in industrial settings.