CogniFuse and Multimodal Deformers: A Unified Approach for Benchmarking and Modeling Biosignal Fusion

Continuously monitored physiological signals carry rich information about the human body and the biological processes happening within. Extracting this information from casually collected biosignal data in activities of daily living holds great potential for real-time monitoring of physical and mental states, but comes with increased difficulty due to the influence of noise and artifacts. Thus, we create CogniFuse, the first publicly available multi-task benchmark for multimodal biosignal fusion in such challenging environments. For many biosignals, especially electrophysiological signals, the information contained in different frequency bands plays a significant role in analyzing the physiological states of the body. Therefore, we introduce a group of novel fusion models, called Multimodal Deformers, that capture multi-level power features as well as long- and short-term temporal dependencies in multimodal biosignal data. In particular, our proposed Multi-Channel Deformer achieves the highest average benchmark score, outperforming all models of comparison. To assure full transparency and reproducibility, and to support future research on multimodal biosignal fusion, all code and data is made publicly available.Clinical relevance- Human biosignals provide important insight into various medical conditions. By advancing the field of multimodal biosignal fusion in activities of daily living, this work contributes to the early detection of complex diseases and impairments outside of highly controlled clinical conditions.