ZUMA: Training-Free Zero-Shot Unified Multimodal Anomaly Detection

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Multimodal anomaly detection (MAD) integrates texture and spatial data for identifying deviations.
• Zero-shot (ZS) settings pose challenges for existing MAD methods due to data constraints.
• Privacy concerns necessitate training-free approaches for MAD.

Purpose of the Study:

• Introduce ZUMA, a training-free framework for zero-shot multimodal anomaly detection (ZS MAD).
• Leverage CLIP's cross-modal capabilities for ZS MAD.
• Address domain gaps and enable flexible anomaly detection across modalities.

Main Methods:

• Propose Cross-Domain Calibration (CDC) to bridge domain gaps and create a hybrid semantic space for 2D/3D data.
• Implement Dynamic Semantic Interaction (DSI) for structural decoupling of anomalies using natural language anchors.
• Develop ZUMA-FT, a fine-tuned variant for enhanced performance with minimal parameters.

Main Results:

• ZUMA achieves state-of-the-art (SOTA) performance on MVTec 3D-AD and Eyecandies benchmarks.
• The training-free ZUMA outperforms existing ZS MAD methods, including training-based approaches.
• ZUMA-FT demonstrates further performance gains with only 6.75M learnable parameters.

Conclusions:

• ZUMA offers a powerful, training-free solution for ZS MAD, adaptable to various data scenarios.
• The framework enables plug-and-play detection of 2D, 3D, or multimodal anomalies.
• ZUMA sets a new benchmark for efficient and effective anomaly detection in complex environments.