Predicting Hospital-associated complications in older adults: methodological considerations

Machine learning is increasingly used to predict post-surgical complications, but predictive performance does not guarantee clinical readiness. Shimizu and colleagues developed a promising model for older adults undergoing tibial fracture surgery using a large Japanese database. However, four methodological pitfalls warrant attention. First, the composite outcome was not validated in this population, and excluding in-hospital deaths introduced survivorship bias, underestimating the true burden of severe complications. Second, days to surgery, a variable not known at admission, created target leakage that inflates performance and undermines the admission-time claim. Third, using the test set for feature selection violated TRIPOD guidelines and likely overstated the scorecard's utility. Fourth, the Super Learner exhibited substantial overfitting, with a 0.110 drop between training (AUC = 0.850) and test (AUC = 0.740) performance, while offering only marginal improvement over simpler models. We suggest redeveloping the model with proper internal validation, excluding leakage-prone predictors, treating death as a competing risk, and prioritizing interpretability when complex ensembles offer limited added value. External validation remains essential before clinical implementation.