DTG: Dual transformers-based generative adversarial networks for retinal 2D/3D OCT image classification

The automated identification of retinal disorders is one of the most popular real-world computer vision applications related to ophthalmology. It has several advantages and can help ophthalmologists identify diseases more accurately. Technically, it represents a retinal data classification problem. With the recent advances in Artificial Intelligence (AI) technologies, Transformer-based architectures have become powerful models commonly used for solving a wide range of tasks such as image classification. In general, even though Transformers have demonstrated excellent performance compared to existing cutting-edge models, they are data-hungry architectures and still need to perform better in automated medical diagnosis applications. In this paper, we propose a deep learning architecture named Dual Transformers-based Generative Adversarial Networks (DTG). It is designed for Optical Coherence Tomography (OCT) data classification. It adopts the Vision Transformer and Multiscale Vision Transformer to encode retinal 2D OCT images (i.e., B-scans) and 3D OCT images (i.e., OCT sequence of B-scans). Then, it employs a proposed Generative Adversarial Networks (GAN) architecture to infer high-quality semantic data representations. Next, it increases the training data by taking advantage of our proposed patient instance-based data augmentation technique. Finally, a weighted classifier analyzes the data and performs the retinal disease classification task. Extensive experiments are carried out on two real-world OCT datasets. The experimental results prove that our proposed approach DTG surpasses several competitors in terms of classification accuracy, precision, recall, f1-score, quadratic weighted kappa, AUC-PR, and AUC-ROC. In particular, it performs better than popular Convolutional Neural Networks and Transformers used for 2D image and 3D OCT image classification. Furthermore, it can improve the performance of several existing works for retinal data classification.