User Perceptions of a Large Language Model-Based Chatbot for Cognitive Reappraisal of Stress: Mixed Methods Feasibility Study

Cognitive reappraisal is a widely studied emotion regulation strategy that helps individuals reinterpret stressful situations in ways that reduce their emotional impact. Digital mental health (DMH) tools often struggle to support this process because scripted templates fail to adapt to the varied and incomplete ways users describe their stressors. Large language models (LLMs) offer opportunities to increase conversational flexibility while preserving structured intervention steps.

This study aimed to examine the feasibility of an LLM-based single-session intervention (SSI) for stress reappraisal. We assessed whether the activity was associated with short-term improvements in stress-related outcomes, and explored the design tensions that arose during user interactions.

We conducted a feasibility study with 100 employees from a large US technology company. Participants completed a structured cognitive reappraisal session delivered by a GPT-4o-based chatbot within Qualtrics. Pre-post measures included situational stress intensity, stress mindset, perceived demand, and perceived resources (all 5-point scales). Paired Wilcoxon signed-rank tests were used with Benjamini-Hochberg correction. To complement self-reports, we analyzed sentiment and stress trajectories across conversation tertiles using a Robustly Optimized BERT Pretraining Approach (RoBERTa; BERT: Bidirectional Encoder Representations from Transformers) sentiment classifier, a RoBERTa stress classifier, and an LLM-based stress rater. We also conducted a thematic analysis of participants' open-ended responses covering stress impact, perspective change, liked and disliked aspects, and improvement suggestions, alongside their conversation logs.

Significant reductions were observed in situational stress intensity (mean 0.29, SD 0.83, r_rb=0.54; P=.002) and significant improvements in stress mindset (mean 1.70, SD 4.37, r_rb=0.44; P=.002). Sentiment and stress classifiers showed consistent declines in negative sentiment and stress from conversation start to end (all omnibus Friedman tests P<.001). Thematic analysis of open-ended responses surfaced several patterns in how participants engaged with the activity: they valued the structured prompts for organizing their thoughts, described shifts in how they interpreted their stressors, used the interaction to identify concrete next steps, and experienced the chatbot as a nonjudgmental space for reflection. Reported design tensions included perceived scriptedness, variable preferences for conversation length, and mixed reactions to AI-driven empathy.

An LLM-enhanced cognitive reappraisal activity showed promise to be delivered as a brief digital intervention and is associated with short-term improvements in situational stress and stress mindset. Participants appreciated the clarity and reflection supported by the structured sequence, while noting important design challenges in balancing structure with conversational naturalness and contextual depth. These findings highlight both the promise and the design constraints of integrating LLMs into DMH interventions.