Validation of a new measure of clinical problem-solving

A review of the literature on the assessment of medical problem-solving by means of written tests reveals serious short-comings. Most important is the low correlation repeatedly found among cases, which suggests the inability of the measures to assess a general problem-solving ability. The literature further suggests that instruments should focus on the brief period of time after the first encounter of a clinical problem and warns against the effects of cueing. Based on these considerations a new measure for the assessment of medical problem-solving was developed. This test, called Simulation of Initial Medical Problem-Solving (SIMP), consists of a number of short case histories, followed by an open-ended question. Reliability analysed by means of generalizability theory proved satisfactory and concurrent validity was established by a significant correlation with a global judgement of performance in a simulated patient encounter. The moderate correlation between cases is interpreted as an acceptable correlation among test items and leads to the conclusion that a reliable and valid test of clinical problem-solving should consist of a substantial number of different cases.