Fusion Strategy Evaluation for Clustering Depression Subtypes Using Multimodal Physiological and Social Data

Wearable and phone sensor data hold great potential for monitoring depression, yet effective integration of these diverse data sources remains challenging. Transforming these complex data into a learned embedding space provides a lower-dimensional representation that preserves essential temporal patterns while capturing the intricate inter-modal relationships. In this study, we evaluate how different fusion strategies for generating multimodal embeddings impact the effectiveness of clustering in identifying depression symptoms. We used a longitudinal dataset integrating physiological and social data such as electrocardiogram, accelerometer, respiration rate, and mobility/Bluetooth interaction data, collected over 35 days. An embedding-based approach using long short-term memory (LSTM) autoencoders was employed to learn latent space representations, followed by the application of K-Means and Gaussian Mixture Models (GMM) clustering algorithms to identify patterns within this learned space. Weekly Beck Depression Inventory-II (BDI-II) scores, held-out during training, served as the ground truth for performance evaluation. A custom metric, the BDI-Variance Ratio Clustering Score (BDI-VRCS), was developed to quantitatively assess clustering efficacy across different embedding spaces. Early fusion implementation with LSTM and GMM achieved the highest BDI-VRCS of 0.3309, outperforming both mid and late fusion strategies (0.112 and 0.132, respectively). This highlights the value of early integration of multimodal data, with social features playing a key role in capturing depressive symptoms.Clinical relevance- This study highlights the potential of integrating physiological and social data using multimodal fusion strategies to enhance depression monitoring and support the development of holistic, data-driven tools for early detection and personalized mental health interventions.