Explainable Suicide Phenotyping from Initial Psychiatric Evaluation Notes Using Reasoning Large Language Models.

Area of Science:

• Clinical Informatics
• Artificial Intelligence in Healthcare
• Psychiatry

Background:

• Clinical phenotyping aids disease understanding by extracting patient symptoms and traits.
• Suicide phenotyping specifically analyzes behavioral and cognitive traits to identify risks and guide interventions.

Purpose of the Study:

• To evaluate the performance of advanced reasoning models (4o, o1, o3-mini) for suicide phenotyping.
• To assess the utility of these models in note-level multi-label classification and clinical justification generation.
• To compare their effectiveness against a fine-tuned GPT-3.5 model.

Main Methods:

• Utilized psychiatric evaluation notes from a safety-net hospital.
• Applied out-of-box reasoning models with in-context learning for classification and generation tasks.
• Implemented novel clinical justification generation.

Main Results:

• Reasoning models achieved comparable or superior performance to GPT-3.5.
• Highest accuracy reached 0.94, with an F1 score of 0.90.
• Demonstrated successful clinical justification generation.

Conclusions:

• Smaller, efficient reasoning models can effectively perform clinical phenotyping.
• The approach offers interpretable and actionable insights for suicide risk assessment.
• This represents a promising advancement in AI-driven psychiatric evaluation.