Multimodal Non-Contact Sensing of Respiration and Movement in Neonates

Preterm neonates represent a vulnerable population which traditional contact-based monitoring devices are not optimized for their small size and complicated physiology. Adhesive sensors and wires can cause infections, discomfort, and impair the delivery of clinical care. Therefore, these most fragile patients could significantly benefit from remote health monitoring. In this study, we establish the foundation for a multimodal device designed for noncontact monitoring of neonates in the Neonatal Intensive Care Unit (NICU) that integrates a video camera and a radar. The device is used to estimate respiratory rate (RR) and detect movement using both unimodal (solely video or radar) and multimodal fusion approaches to combine radar and camera data. We applied a Video Vision Transformer (ViViT) architecture with early, intermediate, and late fusion techniques. Preliminary experiments were conducted using neonatal simulator mannequins and one neonate in the Tufts NICU. For estimation of RR in the neonate including all setting scenarios, early fusion showed the best results with improvements of 14% and 20% in mean absolute error compared to radar and video unimodal baselines, respectively. For movement detection compared to human labeling, the best accuracy and precision for the fusion techniques were higher than 97%. In conclusion, multimodal analysis has the potential to outperform unimodal approaches by improving accuracy against gold standard monitoring, particularly in challenging real-life conditions including motion artifacts and poor lighting.Clinical relevance-Remote sensing of vital signs using video and radar in the NICU offers an alternative to the gold standard adhesive contact sensors routinely used today.