Stress detection through prompt engineering with a general-purpose LLM.

Area of Science:

• Computational linguistics
• Mental health informatics
• Artificial intelligence in healthcare

Background:

• Social media analysis offers potential for mental health monitoring.
• General-purpose large language models (LLMs) show promise but require specialized adaptation for accurate mental health applications.
• Existing methods often lack transparency and require extensive fine-tuning.

Purpose of the Study:

• To develop and evaluate an iterative prompt engineering framework to enhance LLM performance for stress detection in social media.
• To leverage psychologist-informed hints within the prompt engineering process.
• To improve accuracy and reduce false positives in stress detection compared to baseline and domain-specific models.

Main Methods:

• An iterative prompt engineering framework was designed, incorporating psychologist-informed hints.
• The framework was applied to the GPT-4 large language model for stress detection in social media posts.
• Performance was evaluated against baseline zero-shot prompting and domain-specific models, measuring accuracy and false positive rates.

Main Results:

• The prompt engineering approach improved GPT-4's stress detection accuracy from 72% to 89% (a 17% increase).
• A significant 80% reduction in false positives was achieved compared to baseline methods.
• The method outperformed the domain-specific Mental-RoBERTa model by 5% and generated human-readable rationales.

Conclusions:

• Iterative prompt engineering is an effective, resource-efficient strategy for adapting general-purpose LLMs for specialized mental health monitoring tasks.
• The generated rationales enhance transparency and aid mental health professionals in validating model outputs.
• This approach offers a scalable solution for mental health monitoring without the need for costly model fine-tuning.