Cybersecurity and Privacy Risks of Generative AI Mental-Health Chatbots: A Systematic Review and Regulatory Framework

Large language models (LLMs) and other generative artificial intelligence systems are increasingly used in mental health care for psychoeducation, emotional support, screening, and crisis-related interactions. To our knowledge, this is the first structured synthesis explicitly mapping LLM-specific cybersecurity and privacy risks to Software as a Medical Device (SaMD) regulatory frameworks. We aimed to characterize deployment patterns, identify multi-layered risks, and evaluate alignment of reported safeguards with established healthcare governance standards.

A PRISMA-guided systematic review was conducted using PubMed, APA PsycNet, and Google Scholar. After screening eligible records against predefined inclusion criteria, 33 studies were included. Two reviewers independently extracted data on application domains, deployment settings, risk categories, attack surfaces, data sensitivity, and reported or recommended controls.

Generative AI chatbots were most frequently used for therapy or emotional support (13/33, 39.4%), followed by safety evaluation or benchmarking (9/33, 27.3%) and psychoeducation or advice (6/33, 18.2%). Suicide prevention or crisis detection was the most common domain (10/33, 30.3%). Most systems relied on general-purpose LLMs (21/33, 63.6%) and were deployed via consumer-facing platforms (16/33, 48.5%). Key risks included harmful or unsafe outputs, failures in crisis response, exposure of sensitive personal information, and limited transparency. Critically, 78.8% of studies (26/33) were rated high risk for cybersecurity evaluation rigor, indicating that formal adversarial testing and structured threat modeling remain rare.

Current governance frameworks have not fully adapted to generative conversational AI in mental health contexts. Because the therapeutic interface functions as a primary attack surface, single-layer security evaluation (assessing only software validation or content safety in isolation) is inadequate. More comprehensive approaches are needed, including stronger cybersecurity controls, privacy-preserving data practices, and explicit alignment with FDA SaMD guidance, HIPAA, ISO 14971, and the NIST AI Risk Management Framework.