Related Experiment Videos
Learning Gain and User Experience of AI Avatar-Based and Human-Presented Explainer Videos: Prospective Randomized
Mike Reichert1, Thorsten Jungmann2, Ute von Jan1,3
1Department of Digital Medicine, Medical School OWL, Bielefeld University, Bielefeld, North Rhine-Westphalia, Germany.
Background:
Explainer videos are widely used in higher education. With the increasing availability of artificial intelligence (AI)-generated avatars, it remains unclear whether the presentation format-human presenter vs AI avatar-affects learning outcomes and user experience, especially in technologically complex fields.
Objective:
This study aimed to assess the feasibility of a randomized crossover design to investigate learning gain and user experience associated with content-identical explainer videos delivered by either an AI-generated avatar or a human presenter. Exploratory analyses examined the potential differences between the presentation formats.
Methods:
An observer-blinded, prospective randomized crossover feasibility study was conducted with 13 undergraduate engineering students. Participants viewed 2 content-identical explainer videos on fuel cell technology presented by either an AI-generated avatar or a human presenter in a randomized sequence. Learning gains were recorded using a 7-item knowledge test administered at baseline and after the first and second video presentations. User experience was assessed after each video by using the AttrakDiff2 questionnaire. Because there was no washout period and the instructional material was identical in both videos, the second learning phase was vulnerable to carryover and test-retest effects. Consequently, analyses of learning outcomes focused on the initial phase, whereas user experience was examined through pooled comparisons across both conditions.
Results:
Both presentation formats were associated with substantial short-term learning gains. The difference in the learning gain between the AI avatar and human presenter videos was not statistically significant (median newly correct items 5, IQR 3-5.5 vs 4.5, IQR 2.5-5; P=.51; Z=0.66; r=0.183). In contrast, user experience ratings were consistently higher for the human-presented video across all AttrakDiff2 dimensions, with small to medium effect sizes. The AI avatar presentation was generally perceived as neutral.
Conclusions:
This study shows that investigating AI-based explainer videos vs those using a human presenter in classroom settings is feasible and highlights methodological challenges, particularly those related to crossover designs involving content-identical materials. In this small exploratory sample, no significant differences in short-term learning gains were detected between different presentation formats. Nonetheless, participants clearly preferred human presenters in terms of user experience. These results should not be seen as proof of equivalence but rather as a foundation for future research with larger sample sizes, improved study designs, and more sensitive outcome measures.
