Multi-modal Cardio-respiratory Monitoring of Preterm Infants in NICU Using RGB Camera and mmWave Radar.

Area of Science:

• Biomedical Engineering
• Neonatal Monitoring
• Signal Processing

Background:

• Contactless monitoring of vital signs like Heart Rate (HR), Respiratory Rate (RR), and Heart Rate Variability (HRV) is crucial for Neonatal Intensive Care Units (NICUs).
• Existing camera or radar-based systems face performance challenges due to occlusions like incubator glass and shading cloths common in NICU environments.
• Investigating multi-modal fusion of camera and radar data is essential to enhance the reliability of contactless monitoring.

Purpose of the Study:

• To evaluate the performance of individual camera and radar contactless monitoring systems under realistic NICU occlusions (incubator glass, shading cloth).
• To assess the efficacy of camera-radar fusion for improving the accuracy and robustness of HR and RR monitoring in preterm infants.
• To introduce novel methods for signal quality assessment and fusion optimization for multi-modal contactless monitoring.

Main Methods:

• Collected contactless monitoring data (camera and radar) from 50 preterm infants in a NICU setting.
• Quantified the impact of incubator glass and shading cloth occlusions on individual camera and radar HR/RR monitoring performance using Mean Absolute Error (MAE) and Pearson's R-value.
• Developed and validated a Poly-parameter Attention (PPA) model integrating four Signal Quality Indices (SQIs) for optimized multi-modal fusion.

Main Results:

• Incubator glass significantly degraded radar HR monitoring (MAE +5.69 bpm, R -0.26), while shading cloth impacted camera HR monitoring (MAE +3.88 bpm, R -0.19).
• Both occlusions minimally affected RR monitoring accuracy for either modality.
• Camera-radar fusion demonstrated superior performance compared to single modalities, effectively mitigating occlusion challenges and achieving MAE ≤ 5 bpm for both HR and RR using the PPA model.

Conclusions:

• Camera-radar fusion offers a robust solution for contactless HR and RR monitoring in NICUs, overcoming limitations of single-modality systems.
• The proposed PPA model effectively optimizes multi-SQI fusion, significantly outperforming single-modal, single-SQI, and average-weighted fusion approaches.
• This advanced multi-modal approach holds promise for enhancing patient care and monitoring efficiency in Neonatal Intensive Care Units.