A multi-task cross-attention strategy to segment and classify polyps.

Area of Science:

• Medical Imaging Analysis
• Artificial Intelligence in Healthcare
• Gastroenterology

Background:

• Colorectal cancer diagnosis heavily relies on identifying polyps during colonoscopy.
• Accurate polyp characterization, especially malignancy assessment, is crucial but challenging for human experts.
• Existing methods often struggle with precise morphological characterization and segmentation of polyps.

Purpose of the Study:

• To develop a multi-task deep learning model for simultaneous polyp segmentation and malignancy stratification from colonoscopy frames.
• To enhance the accuracy and reliability of polyp detection and characterization in colonoscopy.
• To provide a robust tool for assisting clinicians in diagnosing colorectal cancer.

Main Methods:

• A deep representation using multi-head cross-attention was employed.
• The model was refined with morphological characterization learned from independent maps.
• Validation was performed on the BKAI-IGH dataset (1200 samples) and external datasets.

Main Results:

• Achieved an average Intersection over Union (IoU) of 83.5% for polyp segmentation.
• Reached a recall of 94% for polyp detection and stratification.
• Demonstrated strong generalization capabilities on external datasets, achieving state-of-the-art performance.

Conclusions:

• The proposed multi-task approach effectively integrates polyp segmentation and malignancy stratification.
• The method mimics expert characterization by combining textural and morphological observations.
• This AI-driven strategy shows significant promise for improving early colorectal cancer diagnosis and patient outcomes.