CMFNet: a cross-dimensional modal fusion network for accurate vessel segmentation based on OCTA data

Optical coherence tomography angiography (OCTA) is a novel non-invasive retinal vessel imaging technique that can display high-resolution 3D vessel structures. The quantitative analysis of retinal vessel morphology plays an important role in the automatic screening and diagnosis of fundus diseases. The existing segmentation methods struggle to effectively use the 3D volume data and 2D projection maps of OCTA images simultaneously, which leads to problems such as discontinuous microvessel segmentation results and deviation of morphological estimation. To enhance diagnostic support for fundus diseases, we propose a cross-dimensional modal fusion network (CMFNet) using both 3D volume data and 2D projection maps for accurate OCTA vessel segmentation. Firstly, we use different encoders to generate 2D projection features and 3D volume data features from projection maps and volume data, respectively. Secondly, we design an attentional cross-feature projection learning module to purify 3D volume data features and learn its projection features along the depth direction. Then, we develop a cross-dimensional hierarchical fusion module to effectively fuse coded features learned from the volume data and projection maps. In addition, we extract high-level semantic weight information and map it to the cross-dimensional hierarchical fusion process to enhance fusion performance. To validate the efficacy of our proposed method, we conducted experimental evaluations using the publicly available dataset: OCTA-500. The experimental results show that our method achieves state-of-the-art performance.