DefectSAM: Hierarchically Adapting SAM for Pixel-Wise Surface Defect Detection.

Area of Science:

• Computer Vision
• Machine Learning
• Industrial Inspection

Background:

• The Segment Anything Model (SAM) excels at natural scene segmentation but struggles with industrial defect detection due to weak defect appearance and complex backgrounds.
• Existing methods often fail to capture subtle defect details in industrial settings.

Purpose of the Study:

• To develop a novel hierarchically adapting SAM, named DefectSAM, for precise pixel-wise surface defect detection in industrial images.
• To enhance SAM's generalization capabilities for defect detection tasks.

Main Methods:

• Introduced a learnable feature adaptation component between the encoder and decoder to modulate multilevel features.
• Developed a dual-feature adaptation unit comprising Correlation-Gated Feature Adaptation (CGFA) and Mask-Guided Feature Adaptation (MGFA) modules.
• CGFA integrates convolutional and Transformer features using cross-correlation spatial gating; MGFA uses high-level mask predictions for focused feature adaptation.

Main Results:

• DefectSAM achieved state-of-the-art performance on multiple defect detection datasets (MVTec AD, CrackSeg9k, ZJU-Leaper, Magnetic tile).
• The method demonstrated superior defect detection with a minimal number of learnable parameters.
• Significant improvement in SAM's generalization for industrial defect detection was observed.

Conclusions:

• DefectSAM effectively addresses the limitations of SAM in industrial defect detection by adaptively modulating multilevel features.
• The proposed CGFA and MGFA modules enable robust capture of defect details while suppressing background noise.
• DefectSAM offers a promising, parameter-efficient solution for high-performance industrial surface defect detection.