Development of a Summative Examination with Subject Matter Expert Validation

Area of Science:

• Medical Education
• Assessment and Evaluation

Background:

• Developing effective summative examinations is crucial for evaluating student learning and program effectiveness.
• A systematic approach to examination development can enhance psychometric properties and content validity.

Purpose of the Study:

• To detail the creation, execution, and influence of a summative examination.
• To assess its impact on student learning and programmatic curricular outcomes.

Main Methods:

• A systematic methodology was employed for examination development.
• Psychometric analyses evaluated item reliability, and necessity scoring by subject matter experts assessed content validity.

Main Results:

• Nearly 700 items from 37 faculty members were reviewed.
• Passing standards rose from 45% to 67% (2009-2014) by refining item difficulty and necessity.
• Item discrimination and necessity scores significantly improved over time.

Conclusions:

• The examination successfully measured student and curricular outcomes.
• High faculty engagement fostered a positive assessment culture.
• This examination model offers potential benefits for other educational programs.