DeBERTa-BiLSTM: a multi-label classification model for depression emotions

Multi-label depressive emotion classification remains challenging due to the co-occurrence of multiple mental health-related emotions, implicit linguistic expressions, long-range contextual dependencies, and the presence of both active and passive depressive signals. This paper presents a comprehensive study on multi-label depressive emotion detection using the DepressionEmo dataset, which contains textual instances annotated with eight clinically relevant depression emotions: Anger, Cognitive Dysfunction, Emptiness, Hopelessness, Loneliness, Sadness, Suicide Intent and Worthlessness. The objective is to develop an effective and computationally efficient multi-label classification framework capable of accurately identifying both explicit active and latent passive depressive emotions from text. To address this problem, a broad spectrum of transformer-based and hybrid architectures is evaluated, including BERT, RoBERTa, DistilBERT, T5, BART, and DeBERTa with BiLSTM integration, as well as seq2seq BART and seq2seq RoBERTa-BART models. The proposed DeBERTa-BiLSTM architecture integrates disentangled self-attention for rich contextual representation with a BiLSTM layer for sequential dependency learning and history-state fusion, enabling effective modeling of long-range depressive cues. Experimental results demonstrate that the proposed DeBERTa-BiLSTM model consistently outperforms baseline seq2seq BART, BERT, T5, GAN-BERT, and all other developed variants, achieving an F1-Micro score of 0.83 and an F1-Macro score of 0.80, along with the lowest Hamming Loss (0.15) and the highest Jaccard Index (0.71). The model further achieves micro-precision of 0.81 and micro-recall of 0.85 indicating robust detection of both frequent and minority emotion labels. Runtime analysis shows notable inference efficiency, reducing time per sample by 26.32% at batch size 4 and 21.39% at batch size 32 compared to seq2seq BART. Despite these advantages, the model remains computationally heavier than lightweight transformers, is influenced by the dataset's modest size, and requires further validation across broader mental health domains.