Association between ScholarRx Qmax question bank engagement and NBME CBSE and USMLE Step 1 performance

In recent years, the pass rate of the USMLE Step 1 exam has declined. Failing or delaying Step 1 can cost students time, money, and morale. Undergraduate medical curricula are increasingly supplemented with third-party resources to support successful completion of the curriculum and board exams. Empirical research on the utility of specific third-party resources for student success is needed to inform evidence-based resources and strategies to support students' success on standardized exams. To contribute to filling this gap, the present study examined the relationship between usage of specific components of the third-party resource ScholarRx and student performance on the Comprehensive Basic Science Exam (CBSE) and passing the USMLE Step 1 exam before clerkships began.

Two cohorts of students from a public medical school in the United States were included in the study: one from a Traditional discipline-based curriculum (n = 97) and one from an Integrated Organ System-based (IOS) curriculum (n = 103). ScholarRx data including usage of bricks, practice Qmax questions, flash facts, and express videos; biomedical science course averages; CBSE scores; and Step 1 outcomes were matched, de-identified, and analyzed (Traditional curriculum n = 97; Integrated Organ System curriculum n = 103). Multiple linear regression was performed to assess associations between engagement with specific ScholarRx tools and CBSE performance. Multiple logistic regression was used to investigate statistical predictors of taking and passing Step 1 on time.

Multiple linear regression revealed that student use of ScholarRx practice questions (Qmax; Unstandardized B = 0.004-0.005) and biomedical science grades (Unstandardized B = 1.414-1.610) were significantly, positively associated with CBSE scores (R² = 0.64-0.79, p < 0.05) but bricks, flash facts, and express videos were not. Similarly, logistic regression found that Qmax usage (Unstandardized B = 0.002-0.003) and biomedical science grades (Unstandardized B = 0.342-0.401) were statistically significant predictors of taking and passing Step 1 on time in both curricula (Traditional: x²₍₂₎ = 52.800, p < 0.001; Nagelkerke R² = 0.575; IOS: x²₍₂₎ = 36.706, p < 0.001; Nagelkerke R² = 0.532).

This study found that students' voluntary use of ScholarRx practice questions was associated with higher performance on the CBSE and taking and passing the Step 1 exam on schedule in both a Traditional curriculum and an IOS curriculum.