Assessment of medical students' Surgery knowledge based on Progress Test

Area of Science:

• Medical Education
• Psychometrics
• Surgical Training

Background:

• Progress Testing (PT) is increasingly used in Brazil to evaluate undergraduate knowledge gain.
• Ensuring the validity and reliability of PT items is crucial for accurate interpretation of student performance.
• This study focuses on the psychometric properties of PT items and student performance in surgery.

Purpose of the Study:

• To analyze the psychometric characteristics of Progress Testing items in surgery.
• To evaluate student knowledge gain in surgery from 2017 to 2023.
• To assess the reliability and validity of the assessment tool in a Brazilian context.

Main Methods:

• Utilized Classical Test Theory principles.
• Applied Bloom's taxonomy for item analysis.
• Calculated Cronbach's alpha for reliability.
• Analyzed item difficulty and discrimination indices.
• Tracked student performance over seven academic years.

Main Results:

• Items demonstrated moderate difficulty (0.3-0.4) and fair to good discrimination (0.3-0.4).
• Questions predominantly targeted medium to high levels of Bloom's taxonomy.
• Substantial reliability (Cronbach's alpha >0.6) was maintained throughout the study period.
• Progressive student knowledge gain was observed, with significant increases from the third year, reaching 70-75% by the sixth year.

Conclusions:

• The Progress Testing items in surgery possess adequate psychometric qualities for valid assessment.
• Student knowledge acquisition in surgery progresses throughout the undergraduate curriculum, with notable gains in later years.
• The established measurement framework is replicable for evaluating student learning and improving assessment in other medical disciplines and contexts.