Determining measures of insight and foresight from responses to multiple choice questions

Clinical decisions require insight and foresight. When these are lacking, overconfidence and error can occur.

Multiple choice questions (MCQ) responses were used to estimate insight, as determined by confidence in responses, and foresight, as determined by the safety of incorrect responses.

An MCQ response system was developed which reflected confidence in, and safeness of, responses. An optional examination incorporating this system was offered to medical students at one university.

A total of 372 students completed the study. There was evidence of insight. The proportion of responses that were correct rose from 32% for low confidence to 77% for high confidence. The proportion that was incorrect and not unsafe fell from 34% for low confidence to 7% for high confidence. There was less evidence of foresight. Of the responses that were incorrect, the proportion that was unsafe increased from 52% for low confidence to 68% for high confidence. Measures of insight and foresight increased with experience. Students with poorer knowledge chose more unsafe responses, but with lower confidence. Five students gave multiple highly unsafe responses held with high confidence.

Students had insight but less foresight. Experience and knowledge altered measures of both. This response system identified students who appeared to lack both.