Innovating instructional design through generative AI prompt engineering for health professions educators

Theory-informed and evidence-based educational offerings promote student learning and equity but are time-consuming and require health professions educators to have content expertise in inclusive instructional design. While -generative AI (GAI) offers the potential to overcome these barriers, educators must learn to effectively leverage GAI tools for evidence-based instructional design. In this work, the authors piloted and evaluated a 2-part experiential learning activity to equip educators to effectively engage with GAI for instructional design purposes.

The authors implemented the GAI innovation in the graduate-level "Teaching 100" course (enrollment n = 27) at Harvard Medical School September-November 2023. Educators used GAI to annotate their lesson plans to identify application of, and opportunities to incorporate, evidence-based principles of teaching and learning. The 2-part assignment provided scaffolded instruction on prompt engineering and engaged learners in metacognitive reflection on AI-generated content. The authors evaluated the effectiveness of the GAI innovation according to the Kirkpatrick Model: descriptive analysis of self--reflections evaluated educators' subjective experience (Level 1) and planned behavioral changes (Level 3), while quantification of prompt quality pre-/post-instruction measured educators' learning (Level 2).

Among educators who completed the 2-part assignment (n = 17/27, 62% completion rate), the quality of -educator-generated AI prompts improved following instruction in prompt engineering: pre-instruction 1.4 (1.2) (mean [SD]) vs post-instruction 4.0 (0.8). The difference in means (2.6 points) was statistically significant (P < .0001, 95% CI [1.9, 3.3]). Metacognitive reflections revealed specific actions educators planned to pursue to implement GAI feedback to improve their instructional design. Educators reported that AI-based assignments enhanced their learning.

The authors are developing a stand-alone, interactive GAI tool to be broadly deployed as a faculty development instructional design resource. This future work will yield a scalable solution to the challenge of developing AI literacy among health professions educators to leverage GAI for theory-informed and evidence-based instructional design.