Can generative AI assess PTSD? A clinical validation study of transcribed and direct audio input modalities

Post-traumatic stress disorder (PTSD) remains underdiagnosed due to barriers such as limited access to mental health professionals and resource constraints. While generative artificial intelligence (AI) shows potential in mental health applications, little information is available on its ability to assess PTSD, particularly through direct audio input. We aimed to examine the ability of three AI-based input modalities to support clinical diagnosis of PTSD: Claude 3.5 Sonnet with transcribed input, Gemini 1.5 Pro with transcribed input and Gemini 1.5 Pro with direct audio input.

Participants were adults with trauma histories who completed clinical interviews. Both generative AI and clinicians produced continuous severity scores and binary diagnoses. For diagnostic prediction, AI-generated severity scores were compared with binary clinician diagnoses using receiver operating characteristic curve analysis to compute area under the curve (AUC). Reliability between AI and clinician severity scores was evaluated using intraclass correlation coefficients (ICCs). Accuracy was calculated by comparing AI-generated diagnoses to clinician-rated diagnoses.

The study included 53 participants (mean age=36.9 years, SD=10.6); 47 were female (88.7%). 37 participants (69.8%) met PTSD criteria based on clinician diagnosis. AUCs, ICCs and accuracies (95% CIs) were: Claude transcribed input, 0.94 (0.87 to 1.00), 0.82 (0.71 to 0.92) and 0.89 (0.78 to 0.95); Gemini transcribed input, 0.93 (0.85 to 1.00), 0.83 (0.73 to 0.90) and 0.85 (0.74 to 0.93); Gemini direct audio input, 0.93 (0.84 to 1.00), 0.89 (0.81 to 0.93) and 0.80 (0.68 to 0.90).

Generative AI may support PTSD diagnosis and expand access to care. Future applications should be developed with an emphasis on privacy-preserving deployment.