The pneumonia severity index: Assessment and comparison to popular machine learning classifiers

Area of Science:

• Medical Informatics
• Machine Learning in Healthcare
• Pulmonology

Background:

• Community Acquired Pneumonia (CAP) is a leading cause of death globally.
• Accurate patient severity prognostication is crucial for effective CAP management.
• The Pneumonia Severity Index (PSI) is a widely used clinical tool for stratifying CAP severity.

Purpose of the Study:

• To evaluate and compare the predictive performance of nine machine learning classifiers against the PSI.
• To assess performance using Receiver Operating Characteristic (ROC) Area Under the Curve (AUC) and Average Precision (Precision-Recall AUC) metrics.
• To analyze performance over a large dataset of adult CAP patients.

Main Methods:

• Utilized a dataset of 34,720 adult patient records from 749 US hospitals (2009-2018).
• Compared the predictive accuracy of nine classic machine learning classifiers against the PSI.
• Evaluated performance using ROC AUC and Precision-Recall AUC metrics.

Main Results:

• Machine learning classifiers, particularly Random Forest, demonstrated statistically significant improvements over PSI.
• PR AUC improved by ~33% and ROC AUC by ~6% with machine learning models.
• Machine learning models required only 7 input values, compared to PSI's 20 parameters.

Conclusions:

• Machine learning classifiers offer superior prediction accuracy for CAP severity compared to the PSI.
• While PSI remains valuable, machine learning presents a more accurate and efficient alternative.
• Utilizing multiple performance metrics like PR AUC provides a more comprehensive evaluation of predictive models.