Improving Emergency Department Visit Risk Prediction: Exploring the Operational Utility of Applied Patient Portal...

Area of Science:

• Clinical Informatics
• Natural Language Processing (NLP)
• Healthcare Risk Prediction

Background:

• Patient portal messages offer unique clinical insights into patient behavior and health literacy between appointments.
• Current applications of advanced NLP, including large language models (LLMs), to patient messages are not well understood.
• Effective utilization of patient-generated text data can improve clinical decision-making.

Purpose of the Study:

• To explore methods for integrating patient portal messages into an existing Emergency Department (ED) visit risk prediction model.
• To evaluate the impact of patient message features on the performance of the ED risk prediction model.
• To identify potential improvements in predicting ED visits by leveraging patient communication data.

Main Methods:

• Utilized an existing Emergency Department (ED) visit risk prediction model at Stanford Health Care.
• Incorporated patient message frequencies as a feature into the baseline risk prediction model.
• Evaluated model performance using metrics such as Area Under the Curve (AUC) and F1 score.

Main Results:

• The inclusion of patient message frequencies to the baseline model resulted in an improved AUC of 0.77.
• A significant increase in the F1 score was observed after incorporating patient message frequencies.
• The study demonstrates the value of patient message frequency as a predictive feature.

Conclusions:

• Patient message frequency is a valuable addition to ED visit risk prediction models.
• NLP approaches can effectively leverage patient portal messages to enhance clinical risk assessment.
• Future work will explore incorporating patient message content for further model improvement.