Machine Learning-Based Adjudication of Acute Infection for Diagnostic Classifier Development with Silver-Standard...

Area of Science:

• Medical Informatics
• Machine Learning in Healthcare
• Clinical Decision Support

Background:

• Manual chart review for acute infection adjudication is standard but costly and subjective.
• Accurate training labels are crucial for developing diagnostic classifiers for sepsis.
• Existing methods for acute infection diagnosis are challenging and lack reproducibility.

Purpose of the Study:

• To develop and evaluate a machine learning (ML)-based system for automated adjudication of acute infections.
• To create a reproducible, scalable, and cost-effective method for generating training labels for sepsis diagnostic classifiers.
• To assess the performance and robustness of ML models in automated adjudication.

Main Methods:

• Leveraged an international, multi-cohort dataset with multi-clinician-adjudicated acute infections.
• Benchmarked several off-the-shelf ML models for performance and robustness to missing data.
• Evaluated model calibration and identified minimal feature sets for sustained performance.

Main Results:

• Developed an accurate and well-calibrated ML system based on a logistic regression model.
• The automated system enables low-cost, reproducible, and highly scalable adjudication of acute infections.
• Identified minimal features required for maintaining high performance in future studies.

Conclusions:

• The ML-based system automates the resource-intensive and variable process of chart review for acute infection adjudication.
• This approach enhances the reproducibility and scalability of acute infection adjudication for downstream diagnostic classifier development.
• Automating chart review allows clinicians more time for direct patient care, improving healthcare efficiency.