Competency assessment of an undergraduate program using a third-party, objective pre-post examination.

Area of Science:

• Medical Education
• Competency Assessment
• Program Learning Outcomes

Background:

• Assessing competencies and program learning outcomes presents a leadership challenge in educational programs.
• This case study details a medical education program's strategy for documenting undergraduate competency.
• Utilized a pre-post, third-party, objective testing service for inter-university comparison.

Purpose of the Study:

• To document undergraduate competency attainment in a medical education program.
• To evaluate the effectiveness of a third-party, objective testing service for measuring competency improvement.
• To explore the utility of pre-post testing for assessing program learning outcomes.

Main Methods:

• Students (n=55) completed a pre-test upon program acceptance and a post-test at the end of the final didactic semester.
• Scores and subscores were analyzed using t-tests with Holm-adjusted p-values.
• Multivariate analysis of variance (MANOVA) models were used to evaluate sub-competency difference scores.

Main Results:

• Significant competency improvement was observed across all 12 assessed areas (p < .001).
• Average student improvement was 12.85 points, with the greatest gains in strategic planning and leadership competencies.
• Improvements were consistent regardless of student ethnicity, age, gender, or academic grades.

Conclusions:

• The third-party pre-post testing demonstrated face validity, showing expected performance improvements post-curriculum.
• Repeated testing effectively documents competency attainment, highlighting the insufficiency of single assessment methods.
• The study also identified limitations associated with the specific third-party examination used.