Enhancing OSCE reliability and effectiveness in radiology resident training with long-term systemic evaluation

Area of Science:

• Medical Education
• Radiology Training
• Assessment Methods

Background:

• The Objective Structured Clinical Examination (OSCE) is widely used in medical education for competency assessment.
• Evaluating the long-term effectiveness and reliability of the OSCE in radiology residency is crucial for curriculum development.

Purpose of the Study:

• To assess the long-term systematic effectiveness and reliability of the OSCE in radiology resident training.
• To analyze both subjective feedback and objective examination data from examiners and examinees.

Main Methods:

• A 6-year retrospective observational study (2018-2021, 2023-2024) analyzing OSCE data.
• Collected subjective evaluations from 198 examiners and 818 examinees via questionnaires.
• Analyzed objective data including station scores, difficulty indices, and discrimination indices using statistical methods.

Main Results:

• The OSCE demonstrated stable performance, consistent difficulty, and reliable discrimination across stations over six years.
• Overall final scores remained stable, with strong correlations between station and final scores.
• Examinees perceived higher difficulty than examiners, despite objective data aligning with examiner assessments; standardized training improved consistency.

Conclusions:

• The OSCE is a reliable, valid, and effective assessment tool for radiology residency.
• Multidimensional analysis of subjective and objective data confirms the OSCE's robustness.
• The study highlights the OSCE's role in consistent clinical competency assessment in radiology training.