Enhancing colorectal polyp classification using gaze-based attention networks

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Gastroenterology

Background:

• Colorectal polyps are precursors to colorectal cancer, necessitating accurate endoscopic classification.
• Current deep learning models for polyp classification face challenges in data acquisition, interpretability, and clinical adoption.

Purpose of the Study:

• To develop an improved convolutional neural network (CNN) model for colorectal polyp classification.
• To integrate endoscopist gaze attention information as an auxiliary supervisory signal to enhance CNN performance.

Main Methods:

• Gaze data from endoscopists viewing endoscopic images were collected using an eye-tracker.
• Gaze information was processed and used to supervise the CNN's attention mechanism via an attention consistency module.
• Experiments were conducted using the EfficientNet_b1 model on a dataset of three colorectal polyp types.

Main Results:

• The CNN model with supervised gaze information achieved 86.96% test accuracy, 87.92% precision, 88.41% recall, 88.16% F1 score, and 0.9022 AUC.
• Performance metrics significantly surpassed the model without gaze supervision.
• Class activation maps confirmed that gaze information improved classification accuracy.

Conclusions:

• Integrating endoscopist gaze attention information enhances CNN-based colorectal polyp classification.
• This approach offers a promising solution for improving diagnostic accuracy in medical image analysis.
• The method addresses challenges related to interpretability and clinical acceptance of AI models in endoscopy.