A framework for generalizable neural networks for robust estimation of eyelids and pupils

Deep neural networks (DNNs) have enabled recent advances in the accuracy and robustness of video-oculography. However, to make robust predictions, most DNN models require extensive and diverse training data, which is costly to collect and label. In this work, we seek to improve the codevelop pylids, a pupil- and eyelid-estimation DNN model based on DeepLabCut. We show that performance of pylids-based pupil estimation can be related to the distance of test data from the distribution of training data. Based on this principle, we explore methods for efficient data selection for training our DNN. We show that guided sampling of new data points from the training data approaches state-of-the-art pupil and eyelid estimation with fewer training data points. We also demonstrate the benefit of using an efficient sampling method to select data augmentations for training DNNs. These sampling methods aim to minimize the time and effort required to label and train DNNs while promoting model generalization on new diverse datasets.