Psychometric characteristics of Korean Medical Licensing Examination items from 2012 to 2022 analyzed using the Rasch...

Area of Science:

• Medical Education
• Psychometrics
• Educational Measurement

Background:

• The Korean Medical Licensing Examination (KMLE) is crucial for medical licensure.
• Understanding the psychometric properties of KMLE items is essential for maintaining test quality and fairness.
• Item Response Theory (IRT) models, such as Rasch and 2-parameter logistic (2PL) models, are widely used for this purpose.

Purpose of the Study:

• To characterize the psychometric properties of KMLE items administered between 2012 and 2022.
• To compare the performance of the Rasch model and the 2-parameter logistic (2PL) model in analyzing KMLE item data.
• To descriptively examine items identified as highly difficult under the 2PL model.

Main Methods:

• Item parameters for KMLE items were estimated annually using both Rasch and 2PL models in R.
• Correlations between difficulty estimates from both models were calculated.
• Items with a 2PL difficulty parameter (b) of 2.0 or higher were selected for content analysis.

Main Results:

• Strong correlations (0.70-0.80) were observed between Rasch and 2PL difficulty estimates across years.
• The proportion of highly difficult items (b ≥2.0) under the 2PL model varied between 4.7% and 8.9% annually, without a clear trend.
• The Rasch model generally yielded smaller standard errors compared to the 2PL model.

Conclusions:

• While Rasch and 2PL models produced highly correlated difficulty rankings, absolute estimates differed.
• The Rasch model showed better numerical stability in this context, but model choice depends on fit and intended use.
• Annual differences in item parameters could not be attributed to item disclosure due to separate calibrations and lack of control conditions.