Related Experiment Videos
Using Evidence to Improve Evaluation: A Comprehensive Psychometric Assessment of a SP-Based OSCE Licensing
Carlos A. Brailovsky1, Paul Grand'Maison
1Centre d'évaluation des sciences de la santé and Department of Family Medicine, Université Laval, Quebec, Canada (E-mail: cbrailovsky@cessul.ulaval.ca)
Advances in Health Sciences Education : Theory and Practice
|October 19, 2002
Summary
This study confirms the SP-based OSCE licensing examination in Quebec is valid, reliable, and unbiased. The examination accurately assesses physician competency and impacts medical education.
Area of Science:
- Medical Education
- Psychometric Assessment
- Physician Licensure
Background:
- Bridging psychometric theory with evaluation instruments is complex.
- The SP-based OSCE licensing examination in Quebec, established in 1990, underwent extensive psychometric evaluation.
- This process is crucial for maintaining high standards in medical licensure.
Purpose of the Study:
- To comprehensively review psychometric studies on the SP-based OSCE licensing examination.
- To evaluate the examination's adherence to psychometric assessment theory.
- To ensure the examination's continued quality and validity for physician licensure.
Main Methods:
- Conducted a 5.5-hour OSCE with 26 clinical cases (7-15 min each).
- Assessed reliability, generalizability (across sites, tracks, languages), content validity, concurrent validity, and predictive validity.
- Analyzed potential gender bias using differential item functioning (DIF).
Main Results:
- Confirmed the validity, reliability, and generalizability of examination scores.
- Found no evidence of gender bias through DIF analysis.
- Demonstrated that examination scores predict aspects of physician practice and positively impact teaching/learning.
Conclusions:
- The psychometric evaluation affirmed the high quality of the SP-based OSCE licensing examination.
- The study supports the inferences drawn from examination results for physician licensure.
- Continuous quality improvement and validation of the examination process were ensured.