Exploratory Factor Analysis of a Computerized Case-Based F-Type Testlet Variant

A testlet is a cluster of questions related to a single content area. F-type testlet is a specific type of linear testlets that contains evolving scenarios. There is no study that carried out a factor analysis of an F-type testlet exam in the context of medical education and related disciplines. We aimed to determine to what extent disciplinary domains account for the variability of student performance on an online case-based F-type testlet variant. Final year undergraduate medical students (N = 441) participated in the online exam that consisted of ten case-based F-type testlets. Six of the testlets were pediatrics, four were internal medicine. Exploratory factor analysis was carried out. It revealed that the structure has two factors. Pediatrics testlets loaded factor 1 (loadings between 0.56 and 0.77), and internal medicine testlets loaded factor 2 (loadings between 0.65 and 0.79). The results showed that disciplinary domains account for the variability of the performance on F-type testlets. The results suggest that context specificity still exists in this type of exam. In order to more thoroughly evaluate the clinical reasoning skills of students, medical educators must still ensure that clinical reasoning exams include a comprehensive representation of all relevant content areas.