Fast body part segmentation and tracking of neonatal video data using deep learning

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Neonatal Care

Background:

• Photoplethysmography imaging (PPGI) offers non-contact monitoring for preterm infants in neonatal intensive care units (NICUs).
• Current PPGI methods face challenges with neonatal anatomy and require RGB data, limiting the use of less intrusive near-infrared (NIR) acquisition.
• Automated, real-time region of interest detection is crucial for practical PPGI implementation.

Purpose of the Study:

• To develop a deep learning-based method for real-time video segmentation of neonatal data for PPGI applications.
• To address the limitations of existing segmentation approaches for neonates and enable the use of NIR imaging.
• To create a computationally efficient model capable of processing high-resolution video streams in real time.

Main Methods:

• An encoder-decoder semantic segmentation architecture was augmented with a modified ResNet-50 encoder for improved efficiency.
• The model was pre-trained on public adult datasets and fine-tuned using a comprehensive dataset of neonatal RGB and NIR video recordings.
• Data augmentation techniques were employed to generate virtual NIR data, enhancing performance on NIR imagery.

Main Results:

• The developed method achieved real-time processing speeds (30 fps at 960x576 pixels), a 7.5x reduction in computational time.
• On RGB data, head segmentation achieved 82% intersection over union (IoU) and 88% accuracy, comparable to non-neonatal datasets.
• After data augmentation, NIR head segmentation improved to 62% IoU and 65% accuracy.

Conclusions:

• The deep learning method provides a computationally efficient solution for real-time neonatal video segmentation.
• The approach shows promise for enabling non-contact PPGI monitoring in NICUs, potentially reducing skin complications.
• Further improvements in NIR data segmentation are achievable through targeted data augmentation strategies.