Transformer-Based Deep Learning Framework for Automated Lesion Detection in Capsule Endoscopy: A Comparative Study...

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Gastroenterology

Background:

• Capsule endoscopy analysis is time-consuming and prone to variability.
• Deep learning offers automated lesion recognition but lacks systematic architectural comparisons for multiclass classification.

Purpose of the Study:

• To compare a Vision Transformer (ViT) with Convolutional Neural Network (CNN) architectures (DenseNet121, ResNet50) for multiclass lesion classification in capsule endoscopy.
• To evaluate performance across 21 distinct lesion categories.

Main Methods:

• Merged two public datasets (SEE-AI, Kvasir-Capsule) into a 21-class dataset of ~58,000 capsule endoscopy frames.
• Fine-tuned a pretrained ViT, DenseNet121, and ResNet50 on resized images (224x224).
• Assessed performance using accuracy, precision, recall, F1-score, and AUC, with stratified image-level splitting.

Main Results:

• The Vision Transformer achieved 92.2% accuracy, with a macro-averaged F1-score of 0.92 and AUC of 0.99.
• DenseNet121 achieved 74.0% accuracy (F1-score 0.78, AUC 0.85).
• ResNet50 achieved 38.0% accuracy (F1-score 0.40, AUC 0.55).

Conclusions:

• The Vision Transformer demonstrated superior frame-level classification performance compared to DenseNet121 and ResNet50 on this 21-class dataset.
• Results are based on frame-level splitting and may not represent patient-level generalization.
• Further research, including external validation and workflow studies, is necessary for clinical implementation.