Ensemble of transformers for depression emotion classification

People act on their emotions even in the most rational decision-making mechanisms in their lives. Emotions are powerful motivators that profoundly influence human behavior and social interactions. Human emotions tend to co-occur. Analyzing emotions with this co-occurrence in mind may lead to more accurate insights for addressing various mental health issues. A good example of this is the analysis of depression, which often involves a complex interplay of multiple interrelated emotions rather than a single, isolated feeling. This paper provides a comprehensive analysis of co-occurring emotions in depression by using artificial intelligence methods. We have proposed a transformer-based ensemble model that predicts multiple emotional tendencies associated with depression based on the public DepressionEmo dataset of user posts associated with depression. DistilBERT, RoBERTa, Mental-BERT, Mental-RoBERTa, and DeBERTa are used as pre-trained transformers. The heterogeneous ensemble learning architecture developed using stacking and majority voting methods improves the individual prediction performance of the transformer architectures. Our study is the first to apply the transformer ensemble to the DepressionEmo dataset to identify multiple emotions in an individual's textual psychological posts. Experimental results demonstrate that ensemble-based approaches provide more consistently improved performance compared to individual transformers, particularly in terms of macro-averaged F1 scores under conditions of class imbalance. Among ensemble learning approaches, the highest performance was achieved with Stacking-FFNN, which achieved 0.8121. These ensemble approaches consistently outperformed the strongest individual model, demonstrating the effectiveness of ensemble learning in improving depression emotion classification.