Assesment of Clinical Reasoning in Psychiatric Education: Development of Multiple-Choice Questions with Automatic...

This study aims to evaluate the suitability of the automatic item generation (AIG) for producing Turkish case-based multiplechoice questions (MCQs) in psychiatry.

The study was planned as a descriptive study. In the first stage, topics were determined and a cognitive model was created by subject matter experts. In the second stage, a question template was created, variables were determined, the format of answer options was organized, and two equivalent templates of question content with different combinations were created. In the final stage, questions were generated using Python-based software based on these models. Following the question generation, random samples were selected and evaluated by experienced educators using a structured form.

A total of 1189 questions were generated, with 11 questions sampled for each diagnosis. In the evaluation conducted by experts, six of the questions were deemed appropriate for each parameter, while minor corrections were suggested for five questions. It was stated that all the questions assess clinical reasoning skills rather than factual recall.

The template-based AIG method allows for the rapid and effective production of high-quality questions needed in medical education. The study demonstrated that AIG in the Turkish language for generating MCQs that assess clinical reasoning is applicable in the field of psychiatry. This method enables the production of a large number of questions in a short time, enriched with various combinations. Keyword: Automated item generation, clinical reasoning, medical education, multiple choice question, psychiatry education.