Related Experiment Video
Updated: Jul 5, 2026

Problem-Solving Before Instruction (PS-I): A Protocol for Assessment and Intervention in Students with Different Abilities
Published on: September 11, 2021
Script concordance testing: more cases or more questions?
Robert Gagnon1, Bernard Charlin, Carole Lambert
1CPASS, Faculty of Medicine, University of Montreal, Centre-Ville Station, Box 6128, Montreal, QC H3C3J7, Canada. robert.gagnon@umontreal.ca
Abstract:
Script concordance test (SCT) is a case based assessment format of clinical reasoning in which questions are nested into several cases. Recent results using Q4 format suggest that nested questions contribute more to reliability of measure than cases. The present study aims at documenting variance components associated with SCT cases and nested questions and to determine what are the optimal number and combinations of cases and nested questions. Data from SCT in three different fields are presented. G study and D study methodology are used to estimate variance component and to determine optimal number and combinations of cases and questions. Questions nested into cases contributed a large amount of score variance (more than 70%). D studies with varying samples show that, depending on the reliability of the test, an optimal number of 2-4 questions nested into 15-25 cases represents the best combination. Nested questions contribute to a significant portion of score variance, with the implication that formulation of up to 5 questions per case is an efficient way to optimize the reliability of SCT scores.
Related Concept Videos
Multiple Comparison Tests
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
Comparison Tests
Cochran's Q Test
Testing a Claim about Population Proportion
There are two methods of testing a claim about a population proportion: (1) Using the sample proportion from the data where a binomial distribution is approximated to the normal distribution and (2) Using the binomial probabilities calculated from the data.
The first method uses normal distribution as an approximation to the binomial distribution. The requirements are as follows: sample size is large...
Bonferroni Test
The means of different samples are first paired in all possible combinations.
The null hypothesis of the...
Complementation Tests
Organisms heterozygous for different mutations are crossed pairwise in all combinations. If present on different genes, the mutations can complement each other by providing the missing...