聊天GPT-4o真的可以通过医学科学考试吗? 使用新问题进行实用分析
Philip M Newton1, Christopher J Summers1, Uzman Zaheer1
1Swansea University Medical School, Swansea, Wales, SA2 8PP UK.
Medical science educator
|May 12, 2025
概括
聊天GPT-4o在医疗执照考试中表现出很高的表现,包括新的问题. 它的能力在不受保护的环境中挑战学术完整性,需要安全的考试环境.
科学领域:
- 医疗教育中的人工智能
- 在医疗保健中的自然语言处理.
背景情况:
- 有关ChatGPT在未经检查的在线医疗执照考试中可能存在学术不端行为的担忧.
- 以前的研究表明,ChatGPT的性能可能会被训练数据所膨胀,并受到基于图像的问题的限制.
研究的目的:
- 评估ChatGPT-4o对现有和新型医疗执照考试问题的表现.
- 评估新的问题格式和基于图像的问题对ChatGPT-4o性能的影响.
主要方法:
- 聊天GPT-4o在英国医学执照考试应用知识测试和美国医学执照考试第一步问题上进行了测试.
- 对现有的,重写的和完全新的考试问题进行了评估,包括那些含有图像的考试问题.
主要成果:
- 聊天GPT-4o取得了很高的成绩:英国考试94%,美国考试89.9%.
- 在新的和重写的问题上,表现仍然很强,这表明超出培训数据的适应性.
- 在以图像为基础的,带有文本标签的问题上,观察到性能略有下降.
结论:
- 聊天GPT-4o在医疗许可证考试中展示了先进的功能,即使是新的内容.
- 这些发现强调了需要安全的测试环境,以确保有效的评估和减轻学术不诚实.
- 人工智能模型的持续改进需要对医学教育中的评估策略进行持续的重新评估.
相关概念视频
Positron Emission Tomography
3.9K
Positron emission tomography (PET) is a medical imaging technique involving radiopharmaceuticals — substances that emit short-lived radiation. Although the first PET scanner was introduced in 1961, it took 15 more years before radiopharmaceuticals were combined with the technique and revolutionized its potential.
One of the main requirements of a PET scan is a positron-emitting radioisotope, which is produced in a cyclotron and then attached to a substance used by the part of the body...
One of the main requirements of a PET scan is a positron-emitting radioisotope, which is produced in a cyclotron and then attached to a substance used by the part of the body...
3.9K
The Availability Heuristic
5.9K
A heuristic is a general problem-solving framework (Tversky & Kahneman, 1974). You can think of these as mental shortcuts that are used to solve problems. Different types of heuristics are used in different types of situations, and the impulse to use a heuristic occurs when one of five conditions is met (Pratkanis, 1989):
5.9K
The Scientific Method
59.0K
Research is what makes the difference between facts and opinions. Facts are observable realities, and opinions are personal judgments, conclusions, or attitudes that may or may not be accurate. In the scientific community, facts can be established only using evidence collected through empirical research.
59.0K


