Predicting life expectancy with a long short-term memory recurrent neural network using electronic medical records

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Computational Linguistics

Background:

• Accurate life expectancy prognostication is crucial for end-of-life decision-making and Advance Care Planning.
• Physicians often overestimate life expectancy, potentially missing opportunities to initiate Advance Care Planning.
• This study explores computational methods to improve life expectancy prediction from electronic medical records.

Purpose of the Study:

• To evaluate the efficacy of machine learning and natural language processing in predicting patient life expectancy.
• To compare the predictive performance of these computational models against human physician prognostication.
• To assess the potential of these models in facilitating Advance Care Planning.

Main Methods:

• A supervised machine learning task using a long short-term memory recurrent neural network trained on deceased patients' medical records.
• Model development involved ten-fold cross-validation and evaluation on a held-out test set.
• Comparison included a baseline model without text features, a keyword model utilizing text features, and published physician performance data.

Main Results:

• Physicians and the baseline model achieved 20% accuracy (within a 33% margin of actual life expectancy).
• The keyword model, incorporating text features, achieved 29% accuracy.
• The keyword model showed less overestimation (31%) compared to physicians (63%) in incorrect prognoses.

Conclusions:

• Human prognostication of life expectancy is challenging and prone to overestimation.
• Machine learning and natural language processing present a feasible and promising method for predicting life expectancy.
• This technology has significant potential for real-world application in supporting timely Advance Care Planning.