Reliability and validity studies on modified essay questions

Modified essay questions (MEQs) have been developed at the University of Newcastle to assess clinical problem-solving by first- and second-year medical students. Reliability estimates (given as coefficient alpha [60]) as high as 0.91 were reported for term assessment in 1979 based on MEQs. Lower estimates can be expected as statistical artifacts due to the faculty's criterion-referenced assessment of competence. More appropriate measures of reliability are currently being evaluated. The validity of assessment by MEQ was based on one model of medical problem-solving and another of cognitive skill taxonomies. Internal consistency estimates (coefficient alpha) based on each model yielded some interesting anomalies on the nature of problem-solving which may be reexamined in terms of cognitive preference indexes.