Multimodal physiological signal learning for clinical sleep respiratory event recognition

Automatic recognition of sleep-related breathing events is of great significance for auxiliary sleep disorder screening, clinical interpretation, and long-term physiological monitoring. To address the limitations of existing methods in complementary modeling of multimodal physiological signals, event-related feature aggregation, and imbalanced class learning, this study proposes a multimodal state-space representation learning framework for sleep-related breathing event recognition. The framework takes the neurophysiological signal feature matrix and the respiration-acoustic-related feature matrix as inputs and employs a dual-branch state-space encoder to separately extract sequential dynamic features from different modalities. Furthermore, a cross-state token routing module is designed to realize fine-grained inter-modal information exchange, while an event query prototype fusion module aggregates event-related discriminative features from multimodal sequential representations. During training, class-balanced focal margin loss, sleep-stage auxiliary supervision, and arousal-state auxiliary supervision are jointly employed to improve the model's adaptability to complex sleep segments and imbalanced event distributions. Under patient-level five-fold cross-validation, the proposed method achieves an Accuracy of 88.19%, a Precision of 84.51%, a Recall of 86.28%, and an AUROC of 92.71% on the PSG-Audio public dataset. On the clinical dataset, it achieves an Accuracy of 74.33%, a Precision of 77.51%, a Recall of 84.33%, and an AUROC of 90.51%. These results indicate favorable overall performance of the proposed framework and provide supporting evidence for its ability to learn complementary multimodal physiological representations for sleep-related breathing event recognition.