Related Experiment Video
Updated: May 1, 2026

Author Spotlight: An Efficient and Robust Software for Automated Fusion of Multiple Preclinical Imaging Modalities
Published on: October 27, 2023
Blueprint-Driven Consolidation Improves Preclinical Exam Reliability and Validity
1College of Medicine, University of Florida, Gainesville, Florida, USA.
Background:
Best-practice item writing and blueprint alignment are recommended for medical school assessments, yet few studies examine whether exam redesign improves both psychometric quality and external validity of preclinical block exams in the Step 1 pass/fail era. We therefore examined how consolidating two moderate-reliability gastroenterology (GI) examinations into a single vignette-based exam affected reliability and its relationship with the NBME Comprehensive Basic Science Examination (CBSE).
Approach:
We conducted a retrospective analysis of six cohorts (2019-2025; N = 867) who completed the GI examination(s) and the CBSE. Item statistics and KR-20 reliability coefficients were obtained from the examination platform. Associations between exam average and CBSE total and GI subsection scores were evaluated using Spearman correlations (ρ), and cohort trends were explored using Welch-corrected ANOVA.
Evaluation:
Mean exam average declined modestly (91.6-86.1), whereas CBSE means were stable (≈57-67). Exam reliability rose steadily (KR-20 = 0.51-0.75), with the largest gain after blueprint revision (0.55-0.69). Across all students, exam average correlated moderately with CBSE total (ρ = 0.44, p < 0.001) and GI subsection (ρ = 0.36, p < 0.001). Cohort-specific correlations strengthened over time, increasing from ρ = 0.36 in 2019-2020 to ρ = 0.57 in 2024-2025; cohorts with KR-20 ≥ 0.65 demonstrated stronger predictive validity than earlier cohorts.
Implications:
A clinician-led, vignette-rich, blueprint-driven redesign improved exam reliability and strengthened alignment with national benchmarks despite reduced testing frequency, supporting clinically integrated assessments as a strategy to enhance step-readiness without increasing assessment burden.
Related Concept Videos
Preclinical Development: Overview
Reliability and Validity
Bioequivalence Experimental Study Designs: Repeated Measures, Cross-Over, Carry-Over, and Latin Square Designs
Methods of Documentation V: CBE
In CBE, healthcare professionals establish predefined standards of practice that define what constitutes...
Blind Procedures
Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...

