Biosignal-based screening of depressive symptoms during affective conversations with virtual humans

This study investigates psychophysiological biomarkers of depressive symptoms during socially grounded, ecologically valid casual social interactions. Using an AI-based virtual human, 102 adults were recruited; 98 (51% women; 18-59 years) were analyzed after signal-quality screening (40 with depressive symptoms: PHQ-9 ≥10; 58 healthy controls: PHQ-9 ≤9) during six semi-guided, emotion-eliciting conversations. We recorded electroencephalogram (EEG), heart rate variability, galvanic skin response and eye-tracking data. Unimodal and multimodal voting models were evaluated with nested cross-validation. The emotion-wise multimodal model, trained separately within each of the six narratives, achieved 72% accuracy (AUC = 0.76; specificity = 83%), while EEG alone performed similarly (AUC = 0.75). Other modalities were less informative (AUC = 0.60-0.68). SHAP analyses revealed emotion-dependent, modality-specific patterns underlying predictions. Conversational emotional context improved discrimination over resting baselines, particularly for EEG and the multimodal ensemble, suggesting that virtual humans may reveal depression-related socio-affective signatures beyond passive recordings.