Algorithmic Classification of Psychiatric Disorder-Related Spontaneous Communication Using Large Language Model...

Area of Science:

• Computational linguistics
• Psychiatric diagnostics
• Artificial intelligence in healthcare

Background:

• Language is key to human communication, influenced by thoughts and emotions.
• Psychiatric disorders affect cognitive and emotional processes, altering language use.
• Large language models (LLMs) offer potential for objective language analysis in psychiatry.

Purpose of the Study:

• To explore LLMs for analyzing spontaneous communication to differentiate psychiatric disorders.
• To demonstrate LLM embeddings identify distinct linguistic markers for classifying 7 psychiatric disorders.
• To assess the utility of LLMs in psychiatric condition diagnosis and monitoring.

Main Methods:

• Utilized embeddings from a 7-billion parameter LLM to analyze over 37,000 posts from 7 psychiatric disorder subreddits.
• Trained a cross-validated Extreme Gradient Boosting classifier on LLM embeddings.
• Employed Uniform Manifold Approximation and Projection for visualizing linguistic relationships.

Main Results:

• The classifier achieved a 0.73 average F1-score and a microaverage AUC of 0.95.
• Attention-deficit/hyperactivity disorder (ADHD) posts showed the highest classification AUC (0.97), while borderline personality disorder (BPD) had the lowest (0.89).
• The Generative Representational Instruction Tuning Language Model-7B demonstrated superior performance compared to other state-of-the-art embedding methods.

Conclusions:

• LLMs can objectively analyze language use to distinguish between psychiatric disorders.
• Findings suggest LLMs provide valuable insights into condition-specific linguistic patterns.
• Future research should validate findings in clinical populations and explore comorbidity.