VCF-CLIP: Visual Context-Driven Fine-Grained Prompt Learning for Zero-Shot Anomaly Detection

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Vision-language models (VLMs) have advanced zero-shot anomaly detection (ZSAD), addressing the cold-start problem.
• Existing CLIP-based ZSAD methods rely on manual prompt engineering, limiting their ability to capture diverse anomaly patterns.
• Suboptimal visual-text alignment hinders the performance of current ZSAD approaches.

Purpose of the Study:

• To propose VCF-CLIP, a visual context-driven fine-grained prompt learning framework for CLIP-based ZSAD.
• To overcome limitations of manual prompt design and coarse-grained prompts in existing ZSAD methods.
• To enhance the accuracy and robustness of anomaly detection through improved visual-text alignment.

Main Methods:

• Introduced prompt prototype learning (PPL) to learn unified prompt prototypes for normal and anomalous states, removing manual design.
• Developed a lightweight prompt refinement adapter to dynamically aggregate multi-scale and multi-level visual features.
• Enabled iterative refinement of prompt prototypes for instance-specific, fine-grained anomaly detection.

Main Results:

• VCF-CLIP demonstrated superior performance compared to existing state-of-the-art ZSAD methods.
• Extensive experiments were conducted on 14 benchmarks spanning industrial and medical domains.
• The framework achieved improved visual-text alignment, leading to enhanced anomaly detection accuracy.

Conclusions:

• VCF-CLIP effectively addresses the limitations of manual prompt engineering in CLIP-based ZSAD.
• The proposed prompt prototype learning and refinement strategies enable fine-grained, instance-specific anomaly detection.
• VCF-CLIP represents a significant advancement in zero-shot anomaly detection across various application domains.