Related Experiment Video
Updated: May 5, 2026
03:14
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
1.3K
Depression detection through dual-stream modeling with large language models: a fusion-based transfer learning
Na Wang1,2,3, Weijia Zhang2, Raja Kamil1
1Faculty of Engineering, Universiti Putra Malaysia, UPM Serdang, Serdang, Selangor, Malaysia.
Frontiers in Big Data
|February 20, 2026
Summary
This study introduces a novel parallel transfer learning framework using BERT and T5 fusion for improved depression detection from speech. The method enhances accuracy and precision, offering a privacy-conscious approach to mental health diagnostics.
Area of Science:
- Artificial Intelligence
- Computational Linguistics
- Mental Health Technology
Background:
- Depression diagnosis is crucial but challenged by limited data access and privacy concerns.
- Existing AI models for depression detection struggle with data scarcity and privacy restrictions.
- Large Language Models (LLMs) offer potential but require specialized integration for mental health applications.
Purpose of the Study:
- To develop a privacy-conscious, robust AI framework for depression detection using speech data.
- To leverage the complementary strengths of BERT and T5 through a fusion mechanism for enhanced linguistic analysis.
- To improve the accuracy and reliability of AI-driven depression detection systems.
Main Methods:
- A parallel transfer learning framework integrating BERT and T5 models via a fusion mechanism.
- Utilizing integrated semantic embeddings from both LLMs to capture comprehensive linguistic cues from transcribed speech.
- Employing a dual-branch architecture with 1D CNN and dense networks, followed by a fusion layer for final prediction.
Main Results:
- The proposed BERT and T5 fusion method significantly outperformed baseline models on the E-DAIC dataset.
- Achieved a 91.3% accuracy (3.0% increase), 95.2% precision (6.9% increase), and 90.0% F1-score (1.7% increase).
- Demonstrated the effectiveness of LLM fusion for enhancing depression detection performance.
Conclusions:
- The fusion of BERT and T5 through parallel transfer learning is effective for improving depression detection.
- This approach offers a scalable and privacy-conscious solution for mental health applications.
- Highlights the potential of advanced AI techniques in addressing challenges in mental healthcare.