AttriPrompter: Auto-Prompting With Attribute Semantics for Zero-Shot Nuclei Detection via Visual-Language Pre-Trained...

Area of Science:

• Computational pathology
• Medical image analysis
• Artificial intelligence in medicine

Background:

• Visual-language pre-trained models (VLPMs) excel at object detection in natural scenes.
• Applying VLPMs to histopathology for zero-shot nuclei detection is challenging due to domain differences.
• Existing methods lack effective strategies for automated prompt generation and handling dense nuclei.

Purpose of the Study:

• To investigate the efficacy of the Grounded Language-Image Pre-training (GLIP) model for zero-shot nuclei detection.
• To introduce AttriPrompter, an automated pipeline for generating semantically rich text prompts for nuclei detection.
• To develop a self-trained knowledge distillation framework to address challenges like high nuclei density and overlapping instances.

Main Methods:

• Developed AttriPrompter for attribute generation, augmentation, and relevance sorting to create text prompts.
• Utilized GLIP with generated prompts for initial zero-shot nuclei detection.
• Implemented a self-trained knowledge distillation framework using GLIP predictions as pseudo labels.

Main Results:

• The proposed method demonstrates remarkable performance in label-free nuclei detection.
• Outperformed all existing unsupervised methods for nuclei detection.
• Showcased excellent generality and potential for medical imaging tasks.

Conclusions:

• VLPMs pre-trained on natural data hold significant potential for medical imaging applications.
• AttriPrompter provides an effective, automated approach to prompt engineering for nuclei detection.
• The knowledge distillation framework successfully mitigates challenges associated with dense and overlapping nuclei.