Using a Hybrid of AI and Template-Based Method in Automatic Item Generation to Create Multiple-Choice Questions in...

Area of Science:

• Medical Education
• Artificial Intelligence
• Educational Technology

Background:

• Traditional item writing is labor-intensive.
• Template-based automatic item generation (AIG) relies on expert model development.
• Non-template-based AIG faces accuracy challenges.
• Medical education needs AI support for efficient item generation.

Purpose of the Study:

• To propose and demonstrate a hybrid AIG method for generating item templates in medical education.
• To explore the feasibility of using AI for creating item templates collaboratively with human experts.

Main Methods:

• A mixed-methods, methodological study with proof-of-concept elements.
• Proposed a hybrid AIG method involving structured human-AI interaction.
• Leveraged AI for generating item models (templates) and cognitive models.
• Used two medical multiple-choice questions (respiratory infections, pediatric allergic reactions) for demonstration.

Main Results:

• The hybrid AIG method involves a 7-step process, with the initial 5 steps performed by an expert in a customized AI environment.
• AI demonstrated capability in generating item templates under expert control within 10 minutes.
• Leveraging AI in template development significantly reduced challenges.

Conclusions:

• The hybrid AIG method integrates AI's generative capabilities with expert oversight.
• This approach capitalizes on the strengths and mitigates weaknesses of existing AIG methods.
• It offers a human-AI collaborative model to enhance efficiency in medical education assessment item generation.