Related Experiment Video
Updated: Aug 13, 2025
Deep Neural Networks for Image-Based Dietary Assessment
Published on: March 13, 2021
Conditional Generative Adversarial Networks for Data Augmentation of a Neonatal Image Dataset
Simon Lyra1, Arian Mustafa1, Jöran Rixen1
1Medical Information Technology, Helmholtz Institute for Biomedical Engineering, RWTH Aachen University, 52074 Aachen, Germany.
Insights
This study uses generative adversarial networks to create more training images for monitoring infant vital signs. This artificial data augmentation improves the accuracy of camera-based infant monitoring systems.
Area of Science:
- Medical Imaging
- Artificial Intelligence
- Neonatal Care
Background:
- Current neonatal vital sign monitoring uses sensors that can harm premature infants' skin.
- Camera-based monitoring with deep learning offers a cable-free alternative but requires extensive data.
- Limited neonatal image datasets hinder the validation of deep learning algorithms.
Purpose of the Study:
- To investigate the use of conditional generative adversarial networks (cGANs) for data augmentation in neonatal imaging.
- To create realistic RGB images from edge detection frames of neonates for enlarging training datasets.
- To evaluate the effectiveness of different edge detection methods on cGAN performance.
Main Methods:
- Adapted the Pix2PixHD network architecture for image generation.
- Utilized edge detection frames from neonates as input for the cGAN.
- Optimized hyperparameters and validated generated images using human evaluation and FID score.
Main Results:
- Generated RGB images were evaluated by 30 volunteers, with 23% misidentified as real in a fake-only test.
- In direct comparison, 28% of generated images were rated as more realistic than manually augmented real images.
- Achieved a Frechet Inception Distance (FID) score of 103.82, indicating good image quality.
Conclusions:
- Conditional generative adversarial networks show promise for augmenting limited neonatal image datasets.
- This approach can significantly improve the training and validation of deep learning algorithms for cable-free neonatal monitoring.
- The study demonstrates a viable method to overcome data scarcity in medical AI research for neonates.
Abstract:
In today's neonatal intensive care units, monitoring vital signs such as heart rate and respiration is fundamental for neonatal care. However, the attached sensors and electrodes restrict movement and can cause medical-adhesive-related skin injuries due to the immature skin of preterm infants, which may lead to serious complications. Thus, unobtrusive camera-based monitoring techniques in combination with image processing algorithms based on deep learning have the potential to allow cable-free vital signs measurements. Since the accuracy of deep-learning-based methods depends on the amount of training data, proper validation of the algorithms is difficult due to the limited image data of neonates. In order to enlarge such datasets, this study investigates the application of a conditional generative adversarial network for data augmentation by using edge detection frames from neonates to create RGB images. Different edge detection algorithms were used to validate the input images' effect on the adversarial network's generator. The state-of-the-art network architecture Pix2PixHD was adapted, and several hyperparameters were optimized. The quality of the generated RGB images was evaluated using a Mechanical Turk-like multistage survey conducted by 30 volunteers and the FID score. In a fake-only stage, 23% of the images were categorized as real. A direct comparison of generated and real (manually augmented) images revealed that 28% of the fake data were evaluated as more realistic. An FID score of 103.82 was achieved. Therefore, the conducted study shows promising results for the training and application of conditional generative adversarial networks to augment highly limited neonatal image datasets.
