Subgroup deployment-readiness audit of calibration and clinical utility in perioperative infection prediction

We audited deployment readiness of an end-of-surgery postoperative infection model across clinically relevant subgroups and assessed mitigation of subgroup miscalibration.

We analyzed out-of-fold predictions from 10 719 procedures (2014-2022) with 504 present-on-admission (POA)-coded postoperative infection events across recorded sex, age, and a clinical physiological-reserve proxy. We assessed discrimination, calibration, threshold-specific errors, and decision-curve net benefit, treating t = 0.10 as a reference anchor. We compared no recalibration with common and group-specific cross-fitted isotonic recalibration, evaluated a refit without the Synthetic Minority Oversampling Technique (SMOTE), and performed a temporal-holdout analysis.

Overall area under the receiver operating characteristic curve (AUROC) was 0.876, with similar sex-marginal discrimination in women and men (0.878 vs 0.875). Calibration-in-the-large was -1.02 overall, -0.86 in frail, and -1.34 in prefrail patients; the 6-event nonfrail stratum was not estimable. At t = 0.10, net benefit was positive in frail and negative in prefrail patients in the source cohort, but negative prefrail utility did not persist in the ascertainable-period (2018-2022) and temporal analyses. Common recalibration moved overall calibration-in-the-large from -1.020 to -0.007. Removing SMOTE shifted overall calibration but did not reduce the reserve gradient. Temporal-holdout AUROC was 0.851 with persistent overprediction. Present-on-admission coding was effectively absent before 2018; prevalence-dependent results were therefore additionally reported for the ascertainable 2018-2022 period, and ascertainment differed by physiological reserve.

Discrimination-only or sex-marginal evaluation would have missed deployment-relevant subgroup heterogeneity, while endpoint ascertainment affected prevalence-dependent interpretation.

Joint subgroup evaluation can identify deployment-relevant limitations missed by aggregate metrics. Prospective validation with comparable outcome ascertainment remains necessary before clinical deployment.