在本科医学教育中评估定制聊天机器人:随机交叉混合方法评估性能,实用性和感知
Isaac Sung Him Ng1, Anthony Siu1, Claire Soo Jeong Han1
1Faculty of Life Sciences and Medicine, King's College London, London WC2R 2LS, UK.
Behavioral sciences (Basel, Switzerland)
|September 27, 2025
概括
一个定制的人工智能聊天机器人增强了医学学生对易用性和参与度的看法,但并没有显著改善学术表现. 需要进一步开发以支持更深入的学习和批判性思维.
科学领域:
- 医疗教育 技术 技术 医学教育
- 人工智能在医疗保健教育中的应用
背景情况:
- 大型语言模型 (LLM) 聊天机器人越来越多地用于医学教育.
- 他们的教学影响需要进一步评估.
- 这项研究调查了一个特定领域的聊天机器人对医学学生的表现,感知和参与的影响.
研究的目的:
- 评估定制教育聊天机器人对医学学生学业成绩的影响.
- 与传统学习方法相比,评估学生对聊天机器人的看法.
- 探索聊天机器人对认知参与和学习的影响.
主要方法:
- 一个随机交叉设计,涉及20名一年级医学学生.
- 针对学术任务的自定义聊天机器人 (Lenny AI) 和传统学习方法之间的比较.
- 通过单一最佳答案 (SBA) 问题,任务后调查和重点小组进行评估.
主要成果:
- 学生报告说,他们对聊天机器人的满意度,参与度和感知质量明显更高 (p < 0.05).
- 聊天机器人使用与感知效率和信心的提高相关,但没有改善SBA分数.
- 定性分析表明事实回忆速度更快,但对更高层次推理的支持有限;学生注意到信任,但希望透明度.
结论:
- 教育聊天机器人提高了可用性和学生满意度.
- 在研究的任务中,没有观察到学业表现或更高水平的认知能力的显著改善.
- 未来的聊天机器人设计应该包括自适应性脚手架,透明的采购,并促进更大的教育价值的关键参与.
相关概念视频
01:16Crossover Experiments
4.5K
Crossover experiments, also called the repeated-measurements design, is a study design in which all experimental units are exposed to all treatments in different periods. Crossover experiments are generally used in psychology, the pharmaceutical industry, agriculture, and medicine.
Crossover designs are performed even with smaller sample sizes since the samples can act as their controls. These are better than simple randomized trials since patients are exposed to all the treatments.
Crossover designs are performed even with smaller sample sizes since the samples can act as their controls. These are better than simple randomized trials since patients are exposed to all the treatments.
4.5K
01:15Bioequivalence Experimental Study Designs: Repeated Measures, Cross-Over, Carry-Over, and Latin Square Designs
182
Body:Bioequivalence experimental study designs play a pivotal role in testing the effectiveness of various treatments. Key among these are the repeated measures, cross-over, carry-over, and Latin square designs. In the repeated measures design, each subject receives all treatments, allowing for temporal comparisons. This type of design is useful in reducing variability but requires careful planning to avoid bias.The cross-over design, an economical method, involves sequential administration of...
182
01:20Bioequivalence Experimental Study Designs: Completely Randomized and Randomized Block Designs
228
Body:Bioequivalence experimental study designs are crucial methodologies used in evaluating and comparing the bioavailability of different drug products. These designs are categorized into various types: completely randomized, randomized block, repeated measures, cross and carry-over, and Latin square designs.Completely randomized designs involve randomly allocating treatments to all subjects participating in the experiment. This allocation is achieved by assigning unique random numbers to...
228
01:13Randomized Experiments
8.9K
The randomization process involves assigning study participants randomly to experimental or control groups based on their probability of being equally assigned. Randomization is meant to eliminate selection bias and balance known and unknown confounding factors so that the control group is similar to the treatment group as much as possible. A computer program and a random number generator can be used to assign participants to groups in a way that minimizes bias.
Simple randomization
Simple...
Simple randomization
Simple...
8.9K