Multimodal depression and anxiety severity prediction using convolutional Kepler ResNeXt and lightweight equilibrium BLSTM explainable AI

The severity of depression and anxiety reflects the intensity of symptoms and their effects on daily functioning. Accurate severity prediction may support timely assessment and intervention. This study presents the Convolutional Kepler Mel Lightweight Equilibrium Fusion Graph (CKMLEG), a multimodal framework that predicts depression and anxiety severity using audiovisual data. The framework integrates the CK-ResNeXt network for visual feature extraction, Mel-frequency audio features, and a Lightweight Equilibrium Bidirectional Long Short-Term Memory model with explainable artificial intelligence (LEBLSTM-XAI). We use fine-grained pruning to reduce computational complexity and the Equilibrium Optimization Algorithm (EOA) to optimize model parameters. A three-layer multimodal architecture combines audio and visual representations for severity estimation. The framework is evaluated using the AVEC 2013 and AVEC 2014 datasets. The reported results include a severity prediction accuracy of 98.99%, a recognition accuracy of 99.87%, a mean absolute error (MAE) of 10%, an R² score of 98% and 99%, a mean squared error (MSE) of 40%, and a root mean squared error (RMSE) of 60%. These findings indicate the potential of the proposed framework for audiovisual depression and anxiety severity prediction. Further validation is required to establish its generalizability across different populations and recording conditions.