OTNet: A CNN Method Based on Hierarchical Attention Maps for Grading Arteriosclerosis of Fundus Images with Small...

The severity of fundus arteriosclerosis can be determined and divided into four grades according to fundus images. Automatically grading of the fundus arteriosclerosis is helpful in clinical practices, so this paper proposes a convolutional neural network (CNN) method based on hierarchical attention maps to solve the automatic grading problem. First, we use the retinal vessel segmentation model to separate the important vascular region and the non-vascular background region from the fundus image and obtain two attention maps. The two maps are regarded as inputs to construct a two-stream CNN (TSNet), to focus on feature information through mutual reference between the two regions. In addition, we use convex hull attention maps in the one-stream CNN (OSNet) to learn valuable areas where the retinal vessels are concentrated. Then, we design an integrated OTNet model which is composed of TSNet that learns image feature information and OSNet that learns discriminative areas. After obtaining the representation learning parts of the two networks, we can train the classification layer to achieve better results. Our proposed TSNet reaches the AUC value of 0.796 and the ACC value of 0.592 on the testing set, and the integrated model OTNet reaches the AUC value of 0.806 and the ACC value of 0.606, which are better than the results of other comparable models. As far as we know, this is the first attempt to use deep learning to classify the severity of atherosclerosis in fundus images. The prediction results of our proposed method can be accepted by doctors, which shows that our method has a certain application value.