Holistic decomposition convolution for effective semantic segmentation of medical volume images

Convolutional Neural Networks (CNNs) have achieved state-of-the-art performance in many different 2D medical image analysis tasks. In clinical practice, however, a large part of the medical imaging data available is in 3D, e.g, magnetic resonance imaging (MRI) data, computed tomography (CT) data and data generated by many other modalities. This has motivated the development of 3D CNNs for volumetric image segmentation in order to benefit from more spatial context. Due to GPU memory restrictions caused by moving to fully 3D, state-of-the-art methods depend on subvolume/patch processing and the size of the input patch is usually small, limiting the incorporation of larger context information for a better performance. In this paper, we propose a novel Holistic Decomposition Convolution (HDC), which learns a number of separate kernels within the same layer and can be regarded as an inverse operation to the previously introduced Dense Upsampling Convolution (DUC), for an effective and efficient semantic segmentation of medical volume images. HDC consists of a periodic down-shuffling operation followed by a conventional 3D convolution. HDC has the advantage of significantly reducing the size of the data for sub-sequential processing while using all the information available in the input irrespective of the down-shuffling factors. We apply HDC directly to the input data, whose output will be used as the input to sub-sequential CNNs. In order to achieve volumetric dense prediction at final output, we need to recover full resolution, which is done by using DUC. We show that both HDC and DUC are network agnostic and can be combined with different CNNs for an improved performance in both training and testing phases. Results obtained from comprehensive experiments conducted on both MRI and CT data of different anatomical regions demonstrate the efficacy of the present approach.