在SBA项目中回答模式:学生,GPT3.5和双子座
Olivia Ng1, Dong Haur Phua1,2, Jowe Chu1
1Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, Singapore.
Medical science educator
|May 12, 2025
概括
大型语言模型 (LLM) 在医学考试中显示出重复的答案模式,与学生不同. 像GPT-3.5和Gemini这样的免费LLM在技术问题上不如受过训练的人准确.
科学领域:
- 医学教育 医学教育
- 人工智能在评估中的作用
- 认知科学 认知科学
背景情况:
- 大型语言模型 (LLM) 越来越多地用于生成和回答教育评估.
- 关于在多个代中使用项目统计数据对LLM绩效进行比较的研究有限,特别是在医学教育中.
研究的目的:
- 调查LLM (GPT-3.5,双子座) 在单一最佳答案 (SBA) 问题上的答案模式.
- 为了比较LLM的表现和回答模式与一年级医学学生.
- 评估免费使用的LLM对医学教育评估的适用性.
主要方法:
- 使用了41个SBA问题,这些问题是为一年级医学学生设计的.
- 每个GPT-3.5和Gemini都经过100次代进行了测试.
- 分析了LLM的回答模式,并与学生绩效数据进行了比较.
主要成果:
- 与学生相比,LLMs表现出更多的重复和集群的回答模式.
- 学生在SBA格式内的多选项管理方面表现优于LLM.
- 免费的LLM被发现低于训练有素的学生或技术医疗问题的专家.
结论:
- LLM表现出有问题的重复回答,可能会造成错误.
- 目前免费的LLM在技术医学评估中与上下文解释作斗争.
- 在涉及人工智能的医学教育评估过程中,人类监督仍然至关重要.
相关概念视频
01:29Assessment of the Gastrointestinal System II: Health Perception Pattern
57
Assessing the gastrointestinal (GI) system is a complex process that begins with collecting subjective data. This data, collected through patient interviews, provides crucial insights into the patient's health history, perception patterns, and lifestyle habits, all contributing significantly to GI health.
Health Perception Patterns
Health perception patterns offer valuable insights into a patient's lifestyle habits and how they may impact their GI health. These patterns include:
Health Perception Patterns
Health perception patterns offer valuable insights into a patient's lifestyle habits and how they may impact their GI health. These patterns include:
57
01:22Self-Report Tests of Personality
274
Self-report inventories are objective personality assessments that use multiple-choice items or numbered scales, typically ranging from 1 (strongly disagree) to 5 (strongly agree). They are often called Likert scales after Rensis Likert. These inventories are widely used due to their ease of administration and cost-effectiveness. One of the most prominent examples is the Minnesota Multiphasic Personality Inventory (MMPI), initially developed in the 1940s to assess abnormal personality traits.
274