Related Experiment Video
Updated: Sep 15, 2025
Introduction of an Integrated Pathology Image Management, Artificial Intelligence, and Reporting System
Published on: July 11, 2025
Comparison of applicability, difficulty, and discrimination indices of multiple-choice questions on medical imaging
1Department of Radiology, Eskişehir Osmangazi University, Faculty of Medicine, Eskişehir, Türkiye.
Introduction:
Creating high-quality multiple-choice questions (MCQs) is vital in health education, particularly in fields like medical imaging. AI-based chatbots have emerged as a tool to automate this process. This study evaluates the applicability, difficulty, and discrimination indices of MCQs generated by various AI chatbots for medical imaging education.
Methods:
80 MCQs were generated by seven AI-based chatbots (Claude 3, Claude 3.5, ChatGPT-3.5, ChatGPT-4.0, Copilot, Gemini, Turin Q, and Writesonic) using lecture materials. These questions were evaluated for relevance, accuracy, and originality by radiology faculty, and then administered to 56 students and 12 research assistants. The questions were analyzed using Miller's Pyramid to assess cognitive levels, with difficulty and discrimination indices calculated.
Discussion:
AI-based chatbots generated MCQs suitable for medical imaging education, with 72.5 % of the questions deemed appropriate. Most questions assessed recall (79.31 %), suggesting that AI models excel at generating basic knowledge questions but struggle with higher cognitive skills. Differences in question quality were noted between chatbots, with Claude 3 being the most reliable. The difficulty index averaged 0.62, indicating a moderate level of difficulty, but some models produced easier questions.
Conclusion:
AI chatbots show promise for automating MCQ creation in health education, though most questions focus on recall. For AI to fully support health education, further development is needed to improve question quality, especially in higher cognitive domains.
Implication For Practice:
AI-based chatbots can support educators in generating MCQs, especially for assessing basic knowledge in medical imaging. While useful for saving time, expert review remains essential to ensure question quality and to address higher-level cognitive skills. Integrating AI tools into assessment workflows may enhance efficiency, provided there is appropriate oversight.
More Related Videos
03:14Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
05:47Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
