Development and Validation of a Machine Learning Tool for Plastic Surgery Residency Application Screening

Applications to integrated plastic surgery residency programs have outpaced the growth of available positions. As a result, faculty must review more applications each year. Artificial intelligence provides 1 mechanism for holistic, expedited review of applications. We aimed to develop and validate a machine learning (ML)-based tool to screen residency applications and hypothetically identify which candidates would receive interview invitations.

We retrospectively collected all applications submitted to 1 integrated plastic surgery residency program between 2022 and 2025. We extracted application data and processed it through 4 ML models: XGBoost, Random Forest, CatBoost, and LightGBM. We trained and validated these models on data from the 2022-2024 application cycles and tested them on applications from 2025. The success of the model was judged against the interview decisions of faculty in the 2025 cycle. Measures of model performance included area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve (AUPRC), and F1 score. Shapley additive explanations analysis determined the relative contribution of each variable to the model outcome.

The top performing model was the CatBoost algorithm. The AUROC was 0.92, and the AUPRC was 0.668, indicating excellent performance. With a threshold to maximize F1 score, sensitivity was 95% and specificity was 67%. Total number of publications was the most important feature influencing the interview invitation decision.

We developed and validated a ML algorithm to accurately aid in the selection of residency interviewees. This tool may assist faculty in conducting more holistic and efficient application review.