Colorectal Lesion diagnosis using transformer and deep learning with multiscale feature interface

Colorectal cancer is the third most common malignancy worldwide. Manual screening requires expertise and resources. However, advancements in AI (artificial intelligence) have reduced the computation burden and time. Machine and deep learning have recently been used to diagnose colorectal lesions. The requirement of handcrafted features makes machine learning models expertise-dependent. At the same time, classical CNN (convolutional neural network) miss the global attention of the features. This work presents CDCTNet (colorectal diagnosis convolution transformer network), a hierarchical model for colorectal disease detection. Our model utilized two convolution blocks for the local high-dimensional spatial features from the lesion. In addition, the ViT encoder is used in parallel with the CNN block to provide a global correlation of the feature map. Furthermore, we designed an IEM block for the interaction of the features between the convolution block and ViT encoder to improve the attention on the features. The CDCTNet is evaluated on Kather and Kvasir datasets and obtained a precision and Kappa score of 96.60% and 95.02%, respectively. At the same time, CDCTNet has recall and F1 scores of 98.08% and 97.94%.