Related Experiment Videos
Leveraging Traditional and Generative Artificial Intelligence for Programmatic Decision Making in Faculty Development
Gayle D Haischer-Rollo1, Jessica T Servey, Bizualem Zelelew
1Dr. Haischer-Rollo: Assistant dean for faculty development, associate professor, Department of Pediatrics, School of Medicine, Uniformed Services University, Bethesda, MD. Dr. Servey: Associate dean for faculty affairs, professor, Department of Family Medicine, School of Medicine, Uniformed Services University, Bethesda, MD. Ms. Zelelew: Data analyst II, Office of Faculty Affairs, School of Medicine, Uniformed Services University, Bethesda, MD. Dr. McFate: Director of faculty affairs, assistant professor, Department of Family Medicine, School of Medicine, Uniformed Services University, Bethesda, MD. Ms. Chi: Education coordinator II, Office of Faculty Affairs, School of Medicine, Uniformed Services University, Bethesda, MD. Dr. Duncan: Assistant dean for assessment, assistant professor, Department of Preventive Medicine and Biostatistics, School of Medicine, Uniformed Services University, Bethesda, MD.
Introduction:
Traditional program evaluation in medical education often faces challenges with large data volumes and manual analysis, which can delay data-driven decisions. This article introduces the artificial intelligence (AI)-enhanced program evaluation (AIEPE) framework to address this gap, proposing a novel, integrated approach using both traditional and generative AI to improve efficiency and insight.
Method:
The AIEPE framework is a five-phase, iterative system that augments human evaluators. It begins with Multi-Modal Data Aggregation (Phase 1). Next, a Dual-Stream AI-Powered Analysis (Phase 2) uses traditional AI for sentiment and thematic analysis while generative AI synthesizes narrative summaries. These insights inform AI-Powered Strategic Formulation (Phase 3), which assists in prioritizing issues and drafting action plans. The final phases, Human-in-the-Loop Decision-Making (Phase 4) and Programmatic Implementation and Iteration (Phase 5), ensure continuous improvement with human oversight.
Results:
Applying the AIEPE framework to a faculty development program evaluation, we analyzed 3361 qualitative comments. Sentiment analysis revealed 87.6% positive comments and thematic analysis identified key topics. The dual-stream approach provided a deeper understanding, linking a high no show rate in an AI course to faculty feeling unprepared. This led to a top-priority recommendation to redesign the curriculum.
Discussion:
The AIEPE framework transforms program evaluation from a labor-intensive process into a dynamic engine for continuous improvement. It empowers academic leaders to shift from data collection to architects of data-informed strategy. Although the framework significantly increases efficiency and depth of insight, it highlights the critical need for human oversight to mitigate challenges such as bias and over reliance on technology.
Related Concept Videos
Decision Making: Traditional Method
First, a specific claim about the population parameter is decided based on the research question and is stated in a simple form. Further, an opposing statement to this claim is also stated. These statements can act as null and alternative hypotheses, out of which a null hypothesis would be a...
Decision Making
Automatic decision-making is fast, intuitive, and relies on gut feelings...
Non-equilibrium in the Cell
Purposive Learning
Cognitive Learning
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...