Using Machine Learning to Assess Factors Associated With North American Pharmacist Licensure Examination Performance

Pharmacy graduates' declining performance on the North American Pharmacist Licensure Examination (NAPLEX) remains concerning, but specific student factors related to success remain unclear. Machine learning (ML) algorithms may offer improved methods to identify potentially at-risk students before they take the examination.

Individuals graduating from the University of Kentucky College of Pharmacy in 2024 who passed (n = 93) or failed (n = 30) the NAPLEX on their first attempt were included. Over 20 characteristics related to demographics (eg, age, sex, residence), undergraduate work (eg, university, degree obtained, grade point average [GPA]), performance in the PharmD program (eg, GPA, elective courses taken, etc.), and engagement with NAPLEX preparatory software (RxPrep) were assessed for each student. CLASSify, a web-based platform for analysis of tabular data, was used to assess each of 8 ML algorithms' ability to accurately predict whether a given student passed or failed. The area under the receiver operating curve (AUC-ROC) was primarily used to assess model accuracy. Average absolute SHapley Additive exPlanation (SHAP) value ranks were used to assess feature importance across models.

Four of 8 ML algorithms outperformed logistic regression (AUC-ROC = 0.860), with the highest AUC-ROC in the random forest model (0.930). Across high-performing models, the most important features were the score on the college's high-stakes progression examination (MileMarker 1), engagement with RxPrep, and traditional measures of academic performance.

ML algorithms accurately classified students' NAPLEX first-time performance in this cohort and could offer notable improvements to existing strategies colleges use to identify potentially at-risk students.