Innovating instructional design through generative AI prompt engineering for health professions educators

Area of Science:

• Medical Education
• Artificial Intelligence in Education
• Instructional Design

Background:

• Theory-informed and evidence-based educational offerings are crucial but time-consuming for health professions educators.
• Generative AI (GAI) presents an opportunity to streamline instructional design, but educators need training to use it effectively.
• An experiential learning activity was developed to train educators in leveraging GAI for evidence-based instructional design.

Purpose of the Study:

• To pilot and evaluate a 2-part experiential learning activity designed to train medical educators in using generative AI for instructional design.
• To assess the effectiveness of the intervention in improving educators' ability to engage with GAI tools for creating inclusive and evidence-based educational content.
• To explore educators' experiences and planned behavioral changes regarding GAI integration in their teaching.

Main Methods:

• The intervention was implemented in a graduate-level course at Harvard Medical School with 27 educators.
• Educators used GAI to annotate lesson plans, focusing on evidence-based teaching principles and prompt engineering.
• Evaluation used the Kirkpatrick Model, assessing subjective experience (Level 1), learning via prompt quality (Level 2), and planned behavioral changes (Level 3).

Main Results:

• Prompt quality significantly improved post-instruction (mean 1.4 to 4.0, P < .0001).
• Educators reported enhanced learning and identified specific actions to integrate GAI feedback into their instructional design.
• The intervention demonstrated a 62% completion rate among participants.

Conclusions:

• The pilot intervention effectively improved medical educators' generative AI prompt engineering skills for instructional design.
• Educators found the AI-assisted assignments beneficial for their learning and planned to implement GAI in their practice.
• Future work includes developing a scalable, interactive GAI tool for faculty development in AI literacy and instructional design.