IQE-CLIP: Instance-Aware Query Embedding for Zero-/Few-Shot Anomaly Detection in Medical Domain

Area of Science:

• Computer Vision
• Medical Imaging
• Artificial Intelligence

Background:

• Vision-language models like CLIP have advanced zero-/few-shot anomaly detection (ZFSAD).
• Existing CLIP-based ZFSAD methods often require category knowledge and specific prompts, limiting their ability to distinguish anomalies in joint embedding spaces.
• Current ZFSAD research primarily focuses on industrial applications, with limited exploration in the medical domain.

Purpose of the Study:

• To propose an innovative framework, IQE-CLIP, for ZFSAD tasks specifically tailored for the medical domain.
• To enhance the sensitivity of query embeddings to abnormalities by incorporating both textual and instance-aware visual information.
• To improve the adaptation of CLIP for medical ZFSAD tasks.

Main Methods:

• Developed IQE-CLIP, a framework for medical ZFSAD.
• Introduced class-based and learnable prompting tokens to adapt CLIP for the medical domain.
• Designed an instance-aware query module (IQM) to extract region-level contextual information, generating anomaly-sensitive query embeddings.

Main Results:

• IQE-CLIP achieved state-of-the-art performance on six medical datasets for both zero-shot and few-shot ZFSAD tasks.
• The proposed query embeddings, integrating textual and instance-aware visual data, proved effective in identifying abnormalities.
• The IQM successfully enhanced the sensitivity of embeddings to anomalies.

Conclusions:

• IQE-CLIP offers a significant advancement in medical ZFSAD.
• The framework demonstrates the efficacy of instance-aware query embeddings for anomaly detection in medical imaging.
• IQE-CLIP provides a robust solution for zero-shot and few-shot medical anomaly detection challenges.