LPA-Tuning CLIP: An Improved CLIP-Based Classification Model for Intestinal Polyps.

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computational Pathology

Background:

• Accurate intestinal polyp classification is critical for colorectal cancer prevention.
• Current deep learning models face limitations due to visual similarity and endoscopic variability.
• Single-modal models exhibit efficiency-accuracy trade-offs and neglect pathological semantics.

Purpose of the Study:

• To develop a multimodal framework integrating endoscopic images and structured pathological descriptions for improved polyp classification.
• To address the limitations of single-modal deep learning approaches in capturing pathological semantics.

Main Methods:

• Proposed LPA-Tuning CLIP framework utilizing cross-modal projection matching (CMPM) with ID loss for enhanced intraclass compactness and interclass separation.
• Incorporated structured clinical semantic templates based on WHO diagnostic criteria for consistent pathology annotations.
• Developed medical-aware augmentation techniques to preserve lesion features and minimize domain shifts.

Main Results:

• Achieved 85.8% accuracy and an F1 score of 0.862 on the internal test set for intestinal polyp classification.
• Established a new state-of-the-art performance for this classification task.
• Demonstrated superior performance compared to unimodal and multimodal baselines.

Conclusions:

• The proposed multimodal polyp classification paradigm achieves 85.8% accuracy in three-subtype classification.
• Joint representation learning of endoscopic images and pathology text significantly outperforms unimodal baselines (by 8.7%) and a multimodal baseline (by 4.3%).
• This approach offers a promising direction for enhancing diagnostic accuracy in colorectal cancer screening.