CogniFuse and Multimodal Deformers: A Unified Approach for Benchmarking and Modeling Biosignal Fusion

Area of Science:

• Biomedical Engineering
• Signal Processing
• Machine Learning

Background:

• Physiological signals offer insights into human health but are challenging to analyze in daily life due to noise.
• Existing methods struggle with multimodal biosignal fusion in real-world, non-clinical settings.
• Developing robust methods for analyzing biosignals during activities of daily living (ADL) is crucial for continuous health monitoring.

Purpose of the Study:

• Introduce CogniFuse, the first multi-task benchmark for multimodal biosignal fusion in challenging ADL environments.
• Present novel Multimodal Deformer models designed to capture multi-level power features and temporal dependencies in biosignals.
• Enable transparent and reproducible research in biosignal analysis for improved health monitoring.

Main Methods:

• Created CogniFuse, a benchmark dataset and evaluation framework for multimodal biosignal fusion.
• Developed Multimodal Deformer models, including the Multi-Channel Deformer, to process complex biosignal data.
• Utilized frequency band analysis and captured both short- and long-term temporal dependencies.

Main Results:

• The Multi-Channel Deformer achieved the highest average benchmark score, outperforming all comparison models.
• Demonstrated the effectiveness of Multimodal Deformers in extracting meaningful physiological information from noisy, real-world biosignal data.
• Validated the performance of fusion models in a multi-task learning setting.

Conclusions:

• CogniFuse provides a valuable resource for advancing multimodal biosignal fusion research.
• Multimodal Deformers represent a significant improvement in analyzing physiological states from challenging biosignal data.
• This work facilitates early detection of diseases and impairments through continuous monitoring outside clinical settings.