Related Experiment Videos
Leveraging Traditional and Generative Artificial Intelligence for Programmatic Decision Making in Faculty Development
Gayle D Haischer-Rollo1, Jessica T Servey, Bizualem Zelelew
1Dr. Haischer-Rollo: Assistant dean for faculty development, associate professor, Department of Pediatrics, School of Medicine, Uniformed Services University, Bethesda, MD. Dr. Servey: Associate dean for faculty affairs, professor, Department of Family Medicine, School of Medicine, Uniformed Services University, Bethesda, MD. Ms. Zelelew: Data analyst II, Office of Faculty Affairs, School of Medicine, Uniformed Services University, Bethesda, MD. Dr. McFate: Director of faculty affairs, assistant professor, Department of Family Medicine, School of Medicine, Uniformed Services University, Bethesda, MD. Ms. Chi: Education coordinator II, Office of Faculty Affairs, School of Medicine, Uniformed Services University, Bethesda, MD. Dr. Duncan: Assistant dean for assessment, assistant professor, Department of Preventive Medicine and Biostatistics, School of Medicine, Uniformed Services University, Bethesda, MD.
The AI-enhanced program evaluation (AIEPE) framework streamlines medical education assessment using AI for faster, deeper insights. This approach transforms data analysis, enabling better decision-making and continuous program improvement.
Area of Science:
- Medical Education
- Artificial Intelligence
- Program Evaluation
Background:
- Traditional program evaluation in medical education struggles with large datasets and manual analysis, hindering timely decisions.
- The artificial intelligence (AI)-enhanced program evaluation (AIEPE) framework is introduced to overcome these limitations.
- AIEPE integrates traditional and generative AI for improved efficiency and insight in program assessment.
Purpose of the Study:
- To introduce and describe the AI-enhanced program evaluation (AIEPE) framework.
- To demonstrate how AIEPE can augment human evaluators in medical education program assessment.
- To highlight the potential of AI in transforming program evaluation processes.
Main Methods:
- The AIEPE framework employs a five-phase, iterative process: Multi-Modal Data Aggregation, Dual-Stream AI-Powered Analysis (traditional and generative AI), AI-Powered Strategic Formulation, Human-in-the-Loop Decision-Making, and Programmatic Implementation and Iteration.
- Traditional AI performs sentiment and thematic analysis, while generative AI synthesizes narrative summaries.
- The framework integrates human oversight throughout the evaluation process.
Main Results:
- Application of AIEPE to a faculty development program evaluation analyzed 3361 qualitative comments.
- Sentiment analysis showed 87.6% positive comments, and thematic analysis identified key topics.
- The dual-stream approach revealed that faculty unpreparedness for AI contributed to a high no-show rate in an AI course, leading to a curriculum redesign recommendation.
Conclusions:
- The AIEPE framework revolutionizes program evaluation, shifting it from a labor-intensive task to a driver of continuous improvement.
- It empowers academic leaders to transition from data collection to data-informed strategy development.
- While enhancing efficiency and insight, the framework underscores the necessity of human oversight to manage potential biases and over-reliance on technology.
Related Concept Videos
Decision Making: Traditional Method
First, a specific claim about the population parameter is decided based on the research question and is stated in a simple form. Further, an opposing statement to this claim is also stated. These statements can act as null and alternative hypotheses, out of which a null hypothesis would be a...
Decision Making
Automatic decision-making is fast, intuitive, and relies on gut feelings...
Non-equilibrium in the Cell
Purposive Learning
Cognitive Learning
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...