Prognostic predictions in psychosis: exploring the complementary role of machine learning models

Area of Science:

• Psychiatry
• Computational psychiatry
• Machine learning in healthcare

Background:

• Predicting schizophrenia spectrum disorder trajectories is challenging due to patient variability.
• Machine learning (ML) models show promise for outcome prediction but are not yet in clinical practice.
• Integrating ML models (MLMs) with psychiatrists' predictions is key to improving clinical utility.

Purpose of the Study:

• Compare the prognostic accuracy of psychiatrists and MLMs for short-term remission in first-episode psychosis.
• Investigate if MLMs can enhance psychiatrists' predictive capabilities.

Main Methods:

• 24 psychiatrists predicted remission probabilities based on patient vignettes from the OPTiMiSE trial.
• MLM predictions were shared with psychiatrists to allow for revised estimates.
• Compared predictive accuracy and inter-rater agreement between psychiatrists and MLMs.

Main Results:

• MLM predictive accuracy was low but comparable to psychiatrists for symptomatic remission (0.50 vs. 0.52) and functional remission (0.72 vs. 0.79).
• Inter-rater agreement was low and comparable between psychiatrists and the MLM.
• MLMs did not improve overall predictive accuracy but showed potential for difficult-to-predict cases; however, psychiatrists struggled to identify when to trust MLM output.

Conclusions:

• MLMs have potential as supplementary tools in psychiatric decision-making, especially for challenging cases.
• Current limitations in MLM predictive accuracy and identifying reliance cues restrict their effectiveness.
• Further development is needed to improve MLM utility and facilitate clinical integration, ensuring psychiatrists retain decision-making autonomy.