Assessing assessment: the content and quality of the psychiatry in-training examination

The authors rated questions on the 1979 Psychiatry Resident In-Training Examination for the knowledge area assessed and for quality. The number of questions devoted to each of 35 knowledge areas varied widely, ranging from zero to 52. Thirty-one percent of the 300 questions were judged to be weak. Satisfactory questions and weak questions were not randomly distributed among knowledge areas. Areas such as psychopharmacology and neurology lend themselves to many satisfactory multiple-choice questions, while a substantial number of knowledge areas do not. Psychiatric examinations that employ multiple-choice questions should concentrate only on those areas which can be adequately assessed by this method.