Personalizing Vision-Language Models With Hybrid Prompts for Zero-Shot Anomaly Detection

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Zero-shot anomaly detection (ZSAD) aims for foundational models to detect anomalies across categories without reference images.
• Defining "abnormality" without category-specific "normality" context presents a significant challenge for ZSAD.
• Existing methods often require extensive labeled data or reference images, limiting their applicability.

Purpose of the Study:

• To propose AnomalyVLM, a novel approach for zero-shot anomaly detection that utilizes product standards to define normal and abnormal contexts.
• To overcome the limitations of vision-language models (VLMs) in interpreting complex textual information from standards for anomaly detection.
• To enable flexible and data-free anomaly detection across arbitrary categories.

Main Methods:

• Leveraging generalized pre-trained vision-language models (VLMs) to interpret product standards.
• Generating "hybrid prompts" (including abnormal region descriptions, symbolic rules, and region numbers) from standards to enhance VLM understanding.
• Integrating these hybrid prompts into anomaly detection stages, specifically an anomaly region generator and refiner within VLMs.
• Personalizing VLMs as category-specific anomaly detectors without requiring training data.

Main Results:

• AnomalyVLM demonstrates superior performance and enhanced generalization capabilities on four public industrial anomaly detection datasets.
• The method shows particular effectiveness in detecting texture-based anomalies.
• Experiments on a practical automotive part inspection task validate the approach's real-world applicability.
• The system allows for user control and flexibility in defining and detecting anomalies in novel categories.

Conclusions:

• AnomalyVLM offers a viable alternative to reference-image-based ZSAD by utilizing product standards.
• Hybrid prompts significantly improve VLM comprehension of standards for effective anomaly detection.
• The proposed method provides a flexible, data-efficient solution for anomaly detection across diverse and novel categories.