Automated Assessment of Neonatal Endotracheal Intubation Measured by a Virtual Reality Simulation System

Manual assessment from experts in neonatal endotracheal intubation (ETI) training is a time-consuming and tedious process. Such subjective, highly variable, and resource-intensive assessment method may not only introduce inter-rater/intra-rater variability, but also represent a serious limitation in many large-scale training programs. Moreover, poor visualization during the procedure prevents instructors from observing the events occurring within the manikin or the patient, which introduces an additional source of error into the assessment. In this paper, we propose a physics-based virtual reality (VR) ETI simulation system that captures the entire motions of the laryngoscope and the endotracheal tube (ETT) in relation to the internal anatomy of the virtual patient. Our system provides a complete visualization of the procedure, offering instructors with comprehensive information for accurate assessment. More importantly, an interpretable machine learning algorithm was developed to automatically assess the ETI performance by training on the performance parameters extracted from the motions and the scores rated by experts. Our results show that the leave-one-out-cross-validation (LOOCV) classification accuracy of the automated assessment algorithm is 80%, which indicates that our system can reliably conduct a consistent and standardized assessment for ETI training.