A New Way of Evaluating Effectiveness of URM Summer Pipeline Programs

Many academic medical centers fund educational opportunities (pipeline programs) for students who are underrepresented in medicine (URM). However, there is a sparsity of published literature on pipeline programs and an even smaller body of published literature that investigates program effectiveness.

In a retrospective cohort study (n=12) of the Provost's Summer Mentorship Program-Medicine (SMPM), we evaluated students' rating of program effectiveness, students' rating of the program's impact on their mindsets, and SAT scores. Several program mindsets, including sense of belonging (inclusiveness) in the health professions and connection to mentors in the medical field, reflect common barriers that prevent URM students from pursuing careers in medicine as outlined in pipeline literature. We describe program effectiveness using mean and median ratings of SMPM effectiveness, ratings of mindsets, and SAT scores. We used Wilcoxon Rank Sum to assess pre and post program differences in ratings of mindsets and SAT scores.

SMPM was effective for learners. The overall mean rating for SMPM effectiveness was 4.27. Mindsets for confidence, interest, sense of belonging, college mentorship, and physician mentorship were statistically different from the start to the end of SMPM (p<0.05), with mean improvement of about 34%, 41%, 44%, 180%, and 140% respectively. The mean pre and post SMPM SAT scores as well as 4-month follow-up SAT mean scores were 713 (SD:155), 813 (SD:83), and 1058 (SD:147), respectively. There was a statistically significant difference between all three SAT scores (p<0.05).

In addition to providing educational support, our pipeline program effectively increased students' sense of belonging in the medical field and their connections to physician mentors, which are two common barriers for URM students who are interested in medicine.