Analysis and implications of validation results for a clinical data-based early warning model for geriatric sepsis

Sepsis poses a serious threat to the health and lives of older adults and is difficult to identify accurately in its early stages. To address this issue, our team previously developed an early warning model for sepsis in older adults based on machine learning algorithms and conducted preliminary validation. To further validate the model's performance for clinical use, we collected additional clinical data. In the process, we gained some insights.

We collected clinical data from patients aged 60 years or older admitted to the Emergency Department and Intensive Care Unit of the General Hospital of Ningxia Medical University. We conducted a secondary evaluation of the predictive and generalization capabilities of the geriatric sepsis sepsis early warning model using multiple metrics (including AUC, accuracy, sensitivity, specificity, precision, and F1 score), and analyzed the validation results.

Eight hundred fourteen patients were included in the study, comprising 106 positive patients (sepsis patients) and 708 negative patients (non-sepsis patients). After validation, the model's accuracy was 0.801, sensitivity was 0.764, and AUROC was 0.865.

In the study, although the AUROC of the further validation showed a slight decrease, the early warning model still performed well. The warning model is indeed helpful in identifying geriatric patients patients with sepsis earlier. The performance deviation observed during the validation phase may stem from multiple factors. Overall, machine learning models remain in a stage requiring continuous refinement and development. They are better suited as auxiliary tools in the medical field, necessitating long-term validation and adjustment rather than operating entirely independently.