MentalRoBERTa-Caps: A capsule-enhanced transformer model for mental health classification

Area of Science:

• Natural Language Processing (NLP)
• Computational Psychiatry
• Machine Learning

Background:

• Large Language Models (LLMs) like RoBERTa excel in NLP tasks, including mental illness detection.
• High computational demands of LLMs limit real-time applications in resource-constrained environments.

Purpose of the Study:

• To develop a computationally efficient hybrid model for mental illness detection.
• To balance model performance, interpretability, and computational efficiency.
• To enable real-time mental health monitoring using social media data.

Main Methods:

• Integration of a 6-layer RoBERTa encoder with a capsule network architecture.
• Utilizing dynamic routing for class capsule output generation.
• Employing Local Interpretable Model-Agnostic Explanations (LIME) for model interpretability.

Main Results:

• The hybrid model achieves high accuracy on benchmark mental health datasets.
• Significant reduction in inference time compared to traditional LLMs.
• LIME provides transparent insights into feature contributions and model decisions.

Conclusions:

• The proposed lightweight model offers a practical solution for efficient mental illness detection.
• The hybrid architecture enhances real-world deployability in mental health systems.
• The approach supports explainable AI for mental health predictions.