Does faculty development influence the quality of in-training evaluation reports in pharmacy?

Area of Science:

• Pharmacy education
• Workplace-based learning assessment
• Clinical supervision quality

Background:

• In-training evaluation reports (ITERs) are crucial for student workplace-based learning across health disciplines.
• The quality of ITERs outside of medicine, particularly in pharmacy, remains under-investigated.
• This study addresses the quality of ITERs in pharmacy and the impact of faculty development on supervisor assessment quality.

Purpose of the Study:

• To assess the quality of In-training evaluation reports (ITERs) in a pharmacy program.
• To determine if a faculty development workshop improves the quality of ITERs submitted by clinical supervisors.
• To establish a baseline for ITER quality using the Completed Clinical Evaluation Report Rating (CCERR) scale.

Main Methods:

• A random sample of pharmacy ITERs from 2013-2014 (historical control) and 2015-2016 (intervention and prospective control groups) were analyzed.
• Two independent raters scored ITERs using the nine-item Completed Clinical Evaluation Report Rating (CCERR) scale.
• The intervention group received a faculty development workshop prior to submitting ITERs.

Main Results:

• Mean CCERR scores were significantly lower in the historical control group (37.9) compared to the intervention (22.9) and prospective control (22.7) groups.
• Multiple CCERR domains, including documented evidence and recommendations, scored below acceptable thresholds in all groups.
• The faculty development workshop did not lead to a significant improvement in ITER quality.

Conclusions:

• This is the first study to use the CCERR scale to evaluate ITER quality in pharmacy education.
• Baseline ITER quality in pharmacy is low and was not improved by the faculty development intervention.
• The study identifies areas for improvement in future rater training to enhance the quality of workplace-based assessments.