MDMNet: Multi-dimensional multi-modal network to identify organ system limitation in cardiopulmonary exercise testing

Cardiopulmonary exercise testing (CPET) serves as an integrative and comprehensive assessment tool for cardiorespiratory fitness. In this paper, we present a novel multi-dimensional multi-modal network (MDMNet) to identify functional limitation of organ systems via CPET, which is of great importance in clinical practice and yet a challenging task due to (1) the intricate intra-variable associations, and (2) the significant inter-individual variability.

The proposed model has three compelling characteristics. First, we employ a dedicated embedding strategy for CPET data to map raw inputs into the learned embedding space, facilitating the detection of latent features of physiological variables. Second, we devise a novel multi-dimensional feature extraction module to capture rich features of physiological inputs at different dimensions, which consists of a one-dimensional feature extraction branch unfolding both temporal and spatial patterns of the entire data, and a two-dimensional feature extraction branch based on Gramian Angular Field (GAF) encoding to reveal the complicated temporal correlation relationships between time points within a variable. Third, we integrate these techniques with clinically significant demographic information to establish our MDMNet incorporating multi-dimensional with multi-modal learning, thereby further addressing the issues of complex intra-variable associations and inter-individual variability simultaneously.

We evaluated the proposed method on the publicly available CPET dataset, achieving AUC scores of 0.948, 0.949 and 0.931 for three tasks respectively.

The superiority of our method in discerning inter-individual differences was further demonstrated through partial least squares discriminant analysis, which holds significant potential for automated clinical application of CPET.