Enhancing accuracy and explainability in colorectal lesion classification with attention-supervised Vision...

Area of Science:

• Artificial Intelligence in Medicine
• Computer Vision for Endoscopy
• Medical Image Analysis

Background:

• Accurate colorectal lesion assessment is crucial for treatment and cancer risk stratification.
• The Paris classification aids this assessment but faces inter-observer variability.
• Vision Transformers (ViTs) offer potential but can exhibit diffuse attention, hindering interpretability.

Purpose of the Study:

• To investigate if supervising ViT attention maps with expert annotations improves Paris classification accuracy.
• To enhance the interpretability of ViT models by focusing attention on relevant lesion regions.
• To develop a method that concurrently boosts classification performance and model explainability.

Main Methods:

• Proposed a Lesion-Focused Attention Loss (L_LFA) pretraining objective.
• Used expert polyp bounding boxes to guide ViT attention to annotated lesion areas.
• Applied L_LFA to six ViT architectures, followed by cross-entropy fine-tuning on the SUN dataset for Paris classification.

Main Results:

• Attention-supervised pretraining consistently improved accuracy and lesion-focused attention across ViT models.
• L_LFA enhanced three-class Paris classification accuracy by up to 7 percentage points.
• L_LFA outperformed a Grad-CAM consistency baseline by 5-13 percentage points, with significant association between focused attention and correct predictions.

Conclusions:

• Directly supervising ViT attention with L_LFA effectively leverages expert knowledge.
• This approach jointly improves Paris classification accuracy and spatial interpretability.
• L_LFA demonstrates superior performance compared to Grad-CAM-based explanation regularization.