Three-Stream Convolutional Neural Network for Depression Detection With Ocular Imaging

Depression is a prevalent and severe mental disorder that significantly affects both mind and body, leading to persistent feelings of sadness, despair, and impaired functionality. Diagnosis of depression primarily relies on clinical assessment and observation of symptoms. However, due to the lack of objective indicators, the experience and skills of doctor may lead to misdiagnosis. Current researches indicate that eye movement patterns and pupil dilation can serve as potential biomarkers for emotional and cognitive dysregulation in individuals with depression. However, most studies are based on manually extracted eye movement features, overlooking a significant portion of information available in ocular imaging. This paper proposes Three-Stream Convolutional Neural Network (TSCNN) for detecting depression, leveraging both spatio-temporal information of raw ocular imaging and paradigmatic semantic features. We suggest using optical flow with different sampling intervals to capture temporal features. In the third stream, we employ an encoder to learn semantic information from paradigm images and use it as prior knowledge. Finally, we utilize a fully connected network for classification, achieving an accuracy of 79.3% on our self-collected dataset. The proposed method may demonstrate significant clinical utility in the future.