Using Machine Learning to Assess Factors Associated With North American Pharmacist Licensure Examination Performance

Area of Science:

• Pharmacy education
• Pharmacist licensure examination
• Machine learning in healthcare

Background:

• Declining pharmacy graduate performance on the North American Pharmacist Licensure Examination (NAPLEX) is a concern.
• Identifying at-risk students for NAPLEX success requires improved methods.
• Machine learning (ML) offers potential for enhanced predictive accuracy.

Purpose of the Study:

• To evaluate the effectiveness of ML algorithms in predicting first-time NAPLEX pass/fail outcomes.
• To identify key student factors influencing NAPLEX success.
• To compare ML model performance against traditional logistic regression.

Main Methods:

• Utilized data from 2024 University of Kentucky College of Pharmacy graduates (n=123).
• Assessed over 20 student characteristics including demographics, academic history, and preparatory software engagement.
• Employed 8 ML algorithms via the CLASSify platform, using AUC-ROC for accuracy and SHAP values for feature importance.

Main Results:

• Four ML algorithms surpassed logistic regression (AUC-ROC=0.860).
• The random forest model achieved the highest accuracy (AUC-ROC=0.930).
• Top predictive features included a college progression exam score, RxPrep engagement, and academic performance metrics.

Conclusions:

• ML algorithms demonstrated high accuracy in classifying NAPLEX first-time performance.
• These models can significantly enhance current strategies for identifying students needing support.
• The findings support the integration of ML into pharmacy education for proactive student intervention.