Deep Q-Learning for Gastrointestinal Disease Detection and Classification

Stomach ulcers, a common type of gastrointestinal (GI) disease, pose serious health risks if not diagnosed and treated at an early stage. Therefore, in this research, a method is proposed based on two deep learning models for classification and segmentation. The classification model is based on Convolutional Neural Networks (CNN) and incorporates Q-learning to achieve learning stability and decision accuracy through reinforcement-based feedback. In this model, input images are passed through a custom CNN model comprising seven layers, including convolutional, ReLU, max pooling, flattening, and fully connected layers, for feature extraction. Furthermore, the agent selects an action (class) for each input and receives a +1 reward for a correct prediction and -1 for an incorrect one. The Q-table stores a mapping between image features (states) and class predictions (actions), and is updated at each step based on the reward using the Q-learning update rule. This process runs over 1000 episodes and utilizes Q-learning parameters (α = 0.1, γ = 0.6, ϵ = 0.1) to help the agent learn an optimal classification strategy. After training, the agent is evaluated on the test data using only its learned policy. The classified ulcer images are passed to the proposed attention-based U-Net model to segment the lesion regions. The model contains an encoder, a decoder, and attention layers. The encoder block extracts features through pooling and convolution layers, while the decoder block up-samples the features and reconstructs the segmentation map. Similarly, the attention block is used to highlight the important features obtained from the encoder block before passing them to the decoder block, helping the model focus on relevant spatial information. The model is trained using the selected hyperparameters, including an 8-batch size, the Adam optimizer, and 50 epochs. The performance of the models is evaluated on Kvasir, Nerthus, CVC-ClinicDB, and a private POF dataset. The classification framework provides 99.08% accuracy on Kvasir and 100% accuracy on Nerthus. In contrast, the segmentation framework yields 98.09% accuracy on Kvasir, 99.77% accuracy on Nerthus, 98.49% accuracy on CVC-ClinicDB, and 99.13% accuracy on the private dataset. The achieved results are superior to those of previous methods published in this domain.