Automatic recognition of schizophrenia from facial videos using 3D convolutional neural network

Schizophrenia affects patients and their families and society because of chronic impairments in cognition, behavior, and emotion. However, its clinical diagnosis mainly depends on the clinicians' knowledge of the patients' symptoms. Other auxiliary diagnostic methods such as MRI and EEG are cumbersome and time-consuming. Recently, the convolutional neural network (CNN) has been applied to the auxiliary diagnosis of psychiatry. Hence, in this study, a method based on deep learning and facial videos is proposed for the rapid detection of schizophrenia. Herein, 125 videos from 125 schizophrenic patients and 75 videos from 75 healthy controls based on emotional stimulation tasks were obtained. The video preprocessing included the experiment clips extraction, face detection, facial region cropping, resizing to 500 × 500 pixel size, and uniform sampling of 100 frames. The preprocessed facial videos were used to train the Resnet18_3D. We utilized ten-fold cross-validation, and held-out testing set to evaluate the model with the accuracy, the precision, the sensitivity, the specificity, the balanced accuracy, and the AUC. The Resnet18_3D trained on Film_order achieved the best performance with accuracy, sensitivity, specificity, balanced accuracy, and AUC of 89.00%, 96.80%, 76.00%, 86.40% and 0.9397. The neural network model indeed recognizes healthy controls and schizophrenic patients through the changes in the area of the face. The results show that facial video under emotional stimulation can be used to classify schizophrenic patients and help clinicians with diagnosis in the clinical environment. Among the different types of stimuli, the video stimuli with fixed emotional order showed the best classification performance.