Use of simplex learning index to identify a course deficit

Difficulty index and its derivative performance index (PFI) have been commonly used to measure student learning outcomes. However, these indexes have a high volatility and low sensitivity. This work has established the simplex learning index (SLI), which has a low volatility and high sensitivity. To construct SLI, students were divided into two groups based on their scores on the whole quiz. The item SLI and lecture SLI were derived from the results of students in the high-scoring group. Exam results from nine cohorts of medical students in two phases of learning were analyzed. The volatility, measured by the ratio between the standard deviation and the mean, was >65% lower in SLI than in PFI. Using the lecture SLI as a metric, one lecture B22 (Metabolism of Amino Acids), was identified that had an average SLI of 0.66 in earlier four student cohorts in phase 1 learning. Two major changes were made on the lecture, lecture organization and the delivery method, in phase 2. Students from recent five cohorts in phase 2 had an average SLI of 0.84, which was 26.6% higher than that in phase 1 (P < 0.02). In contrast, when PFI was used, the change was only 13.46% and insignificant (P = 0.29). In the same period, implementation of the same delivery method did not yield significant changes in learning outcomes in lecture B24 (Metabolism of Nucleotides). Taken together, this work shows that SLI is a better indicator for learning outcomes and suggests that lecture reorganization is the key to improved student learning.