Comparing Generative Artificial Intelligence and Mental Health Professionals for Clinical Decision-Making With...

Area of Science:

• Artificial Intelligence in Healthcare
• Clinical Psychology
• Mental Health Research

Background:

• Trauma exposure is common and linked to health issues.
• Healthcare professionals may exhibit diagnostic overshadowing bias, affecting trauma-exposed individuals.
• Generative artificial intelligence (GAI) is increasingly used in healthcare, but its biases are unexamined.

Purpose of the Study:

• To assess trauma-related diagnostic overshadowing in frontier GAI models.
• To compare GAI models' bias to that of mental health professionals (MHPs).

Main Methods:

• 232 MHPs and multiple GAI models (Gemini, ChatGPT, Claude, Llama) evaluated vignettes with and without trauma history.
• Participants and GAI models made diagnostic and treatment preference decisions.
• Statistical analyses (Mann-Whitney U, chi-square) compared decision-making.

Main Results:

• GAI models showed some trauma-related diagnostic overshadowing bias, particularly with sexual trauma.
• GAI models exhibited significantly less bias than MHPs across tasks.
• GAI models were more accurate than MHPs in diagnosis and treatment selection for OCD and substance use disorder vignettes with trauma.

Conclusions:

• GAI models demonstrate trauma-related diagnostic overshadowing bias, though less than MHPs.
• Bias levels in GAI vary by task and model.
• Understanding and mitigating GAI biases is crucial for responsible implementation in mental health care.