An Ensemble Approach Integrating Retrieval-Augmented Large Language Models and Boosting Algorithms for Enhanced...

Area of Science:

• Computational psychiatry
• Medical informatics
• Machine learning in healthcare

Background:

• Accurate phenotyping of catatonia from electronic health records (EHRs) is essential for large-scale studies.
• Traditional machine learning methods may lack the nuanced understanding required for complex clinical conditions like catatonia.

Purpose of the Study:

• To develop and evaluate an ensemble phenotyping algorithm for catatonia using retrieval-augmented generation (RAG) large language models (LLMs) and boosting algorithms.
• To assess the interpretability and performance of the RAG-LLM component in capturing complex catatonia features from clinical notes.

Main Methods:

• An ensemble model combining RAG-LLMs and boosting algorithms was developed.
• The model was applied to EHR data from 3.5 million individuals (2006-2017).
• Performance was evaluated using Area Under the Receiver Operating Characteristic Curve (AUROC).

Main Results:

• The ensemble model achieved an AUROC of 0.709.
• The boosting algorithm alone achieved a slightly higher AUROC of 0.713.
• The RAG-LLM component significantly improved interpretability by identifying complex features, including those from the Bush-Francis Catatonia Rating Scale, directly from clinical notes.

Conclusions:

• RAG-LLMs show potential in capturing nuanced contextual information for complex phenotyping tasks, even when overall performance is similar to traditional methods.
• The ensemble approach offers a balance between classification performance and enhanced interpretability for catatonia phenotyping.
• This methodology can advance the use of large-scale EHR data for studying conditions like catatonia.