Bigger But Not Better: Small Neural Language Models Outperform Large Language Models in Detection of Thought...

Area of Science:

• Computational linguistics
• Psychiatry
• Artificial intelligence

Background:

• Disorganized thinking is a key diagnostic marker for schizophrenia-spectrum disorders.
• Large language models (LLMs) show potential in assessing thought disorder severity but face deployment challenges like cost and privacy.
• Smaller neural language models offer a potential alternative for clinical applications.

Purpose of the Study:

• To investigate the efficacy of smaller neural language models in detecting positive formal thought disorder.
• To compare the performance of smaller models against larger models using perplexity measurements.
• To assess the generalizability of findings across different speech sample types.

Main Methods:

• Utilized sliding window-based perplexity measurements on speech transcripts.
• Compared the sensitivity of various smaller neural language models to linguistic markers of formal thought disorder.
• Evaluated model performance on audio diaries and clinical interview speech samples from individuals with psychotic symptoms.

Main Results:

• Smaller neural language models demonstrated higher sensitivity in detecting formal thought disorder compared to larger models.
• Optimal detection capability was observed within specific model sizes and context lengths, challenging the 'bigger is better' assumption.
• Findings were consistent across diverse speech sample types, indicating robust performance.

Conclusions:

• Smaller neural language models are effective and potentially superior alternatives for detecting formal thought disorder.
• These findings pave the way for developing efficient, cost-effective, and privacy-preserving screening tools for psychosis.
• The study suggests a paradigm shift in the application of language models in clinical settings, prioritizing efficiency and accessibility.