Introducing progress testing in McMaster University's problem-based medical curriculum: psychometric properties and...

Area of Science:

• Medical Education
• Health Professions Education
• Assessment in Medical Training

Background:

• Progress testing, like the Quarterly Profile Examination, offers summative and formative insights into student knowledge.
• The Personal Progress Index (PPI) was developed to evaluate student progress without negatively impacting learning.
• This study details four years of experience with the PPI at McMaster University Faculty of Health Sciences.

Purpose of the Study:

• To evaluate the reliability, validity, and impact on learning of the Personal Progress Index (PPI).
• To assess the PPI's effectiveness in tracking medical student knowledge progression over time.

Main Methods:

• A 180-item multiple-choice progress test (PPI) covering all medical disciplines was administered thrice annually to all medical students.
• Data from entering classes of 1992-1995 (400 students) were analyzed for reliability and validity.
• Predictive validity was assessed against performance on the Medical Council of Canada licensing examination.

Main Results:

• Internal reliabilities ranged from .46 to .63 across administrations.
• The PPI demonstrated strong construct validity, showing significant differences between classes and progress within classes.
• Predictive validity for the licensing exam increased from .12 to .60 as students progressed through the program.

Conclusions:

• The PPI effectively measures student growth throughout the MD program, aligning with its intended purpose.
• Increased student self-referral for remediation and identification of persistent learning difficulties were observed benefits.
• Consistent performance patterns aid in identifying students with significant and ongoing academic challenges.