Development of knowledge tests for multi-disciplinary emergency training: a review and an example

Area of Science:

• Medical Education
• Assessment and Evaluation
• Obstetric Anesthesia

Background:

• Developing and validating knowledge tests in multi-disciplinary postgraduate settings is challenging.
• Literature on written test development in such environments is limited.
• This study addresses the need for robust assessment tools in specialized postgraduate training.

Purpose of the Study:

• To describe the development and evaluation process of a multiple-choice question (MCQ) test.
• To create an assessment tool for a multi-disciplinary postgraduate training program in obstetric-anesthesia emergencies.
• To establish acceptable validity and reliability for the developed MCQ test.

Main Methods:

• A multi-disciplinary working committee and 28 participants were involved in item development.
• The process included stepwise item development, revisions, and statistical analysis.
• Testing involved face validity, content validity, construct validity, item-total correlation, and reliability analysis.

Main Results:

• The final 40-item MCQ test achieved acceptable content validity.
• Good construct validity was evidenced by significant score differences between professional levels.
• Reliability was acceptable with a Cronbach's alpha of 0.83.

Conclusions:

• The developed MCQ test demonstrated acceptable content and construct validity and reliability.
• The methodology provides a template for developing knowledge tests in multi-disciplinary postgraduate training.
• This approach can enhance the quality of educational assessments in specialized healthcare fields.