Multimodal retinal vessel segmentation from spectral-domain optical coherence tomography and fundus photography

Segmenting retinal vessels in optic nerve head (ONH) centered spectral-domain optical coherence tomography (SD-OCT) volumes is particularly challenging due to the projected neural canal opening (NCO) and relatively low visibility in the ONH center. Color fundus photographs provide a relatively high vessel contrast in the region inside the NCO, but have not been previously used to aid the SD-OCT vessel segmentation process. Thus, in this paper, we present two approaches for the segmentation of retinal vessels in SD-OCT volumes that each take advantage of complimentary information from fundus photographs. In the first approach (referred to as the registered-fundus vessel segmentation approach), vessels are first segmented on the fundus photograph directly (using a k-NN pixel classifier) and this vessel segmentation result is mapped to the SD-OCT volume through the registration of the fundus photograph to the SD-OCT volume. In the second approach (referred to as the multimodal vessel segmentation approach), after fundus-to-SD-OCT registration, vessels are simultaneously segmented with a k -NN classifier using features from both modalities. Three-dimensional structural information from the intraretinal layers and neural canal opening obtained through graph-theoretic segmentation approaches of the SD-OCT volume are used in combination with Gaussian filter banks and Gabor wavelets to generate the features. The approach is trained on 15 and tested on 19 randomly chosen independent image pairs of SD-OCT volumes and fundus images from 34 subjects with glaucoma. Based on a receiver operating characteristic (ROC) curve analysis, the present registered-fundus and multimodal vessel segmentation approaches [area under the curve (AUC) of 0.85 and 0.89, respectively] both perform significantly better than the two previous OCT-based approaches (AUC of 0.78 and 0.83, p < 0.05). The multimodal approach overall performs significantly better than the other three approaches (p < 0.05).