聊天GPT可以通过AHA考试:开放式问题优于多选题格式
Lingxuan Zhu1, Weiming Mou2, Tao Yang2
1Department of Urology, Renji Hospital, Shanghai Jiao Tong University School of Medicine, 160 Pujian Road, Shanghai 200127, China; The First Clinical Medical School, Southern Medical University, 1023 Shatai South Road, Guangzhou, 510515 Guangdong, China.
Resuscitation
|June 22, 2023
概括
聊天GPT最初在美国心脏协会 (AHA) 的考试中失败,但得到了显著的改善. 通过生成多个答案和使用开放式问题,ChatGPT能够高准确度通过基本生命支持 (BLS) 和高级心血管生命支持 (ACLS) 考试.
科学领域:
- 医疗教育中的人工智能
- 在医疗保健中的自然语言处理.
背景情况:
- 美国心脏协会 (AHA) 的基本生命支持 (BLS) 和高级心血管生命支持 (ACLS) 认证对医疗保健专业人员至关重要.
- 在医学认证考试中评估高级AI模型 (如ChatGPT) 的功能对于了解其潜在的应用和局限性至关重要.
研究的目的:
- 评估ChatGPT在AHA的BLS和ACLS认证考试中的表现.
- 调查改善ChatGPT对这些医学检查的准确性的方法.
主要方法:
- 最初的测试涉及每个问题只有一种答案,揭示了BLS和ACLS考试中的失败.
- 随后的测试产生了每个问题的三个答案,评估了BLS考试的准确性.
- 进一步的改进包括将错误回答的问题重写为开放式提示,重新评估BLS和ACLS考试的准确性.
主要成果:
- 聊天GPT最初在BLS和ACLS考试中都失败了,因为每个问题只能产生一个答案.
- 每个问题有三个答案,ChatGPT在BLS考试中获得了84%的准确率.
- 重写错误的答案作为开放式问题提高了准确度,达到96%的BLS和92.1%的ACLS,使ChatGPT能够通过这两次考试.
结论:
- 聊天GPT在BLS和ACLS考试中的表现高度依赖于响应生成策略.
- 修改提示结构,例如使用开放式问题,在医疗认证环境中显著提高了AI准确性.
- 人工智能模型对医学教育和评估有希望,但需要仔细的快速工程才能获得最佳结果.
相关概念视频
Surveys
14.8K
Often, psychologists develop surveys as a means of gathering data. Surveys are lists of questions to be answered by research participants, and can be delivered as paper-and-pencil questionnaires, administered electronically, or conducted verbally. Generally, the survey itself can be completed in a short time, and the ease of administering a survey makes it easy to collect data from a large number of people.
14.8K
Multiple Comparison Tests
3.9K
Multiple comparison test, abbreviated as MCT, is a post hoc analysis generally performed after comparing multiple samples with one or more tests. An MCT will help identify a significantly different sample among multiple samples or a factor among multiple factors.
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
3.9K
Social Loafing
34.8K
Another way in which a group presence can affect performance is social loafing—the exertion of less effort by a person working together with a group. Social loafing occurs when our individual performance cannot be evaluated separately from the group. Thus, group performance declines on easy tasks (Karau & Williams, 1993). Essentially individual group members loaf and let other group members pick up the slack. Because each individual’s efforts cannot be evaluated,...
34.8K
Group Design
9.0K
The most basic experimental design involves two groups: the experimental group and the control group. The two groups are designed to be the same except for one difference— experimental manipulation. The experimental group gets the experimental manipulation—that is, the treatment or variable being tested—and the control group does not. Since experimental manipulation is the only difference between the experimental and control groups, we can be sure that any differences between...
9.0K
One-Way ANOVA
8.0K
One-way ANOVA analyzes more than three samples categorized by one factor. For example, it can compare the average mileage of sports bikes. Here, the data is categorized by one factor - the company. However, one-way ANOVA cannot be used to simultaneously compare the sample mean of three or more samples categorized by two factors. An example of two factors would be sports bikes from different companies driven in different terrains, such as a desert or snowy landscape. Here, two-way ANOVA is used...
8.0K
Assessment of the Cardiovascular System I: Subjective Data
371
A thorough health history and physical assessment are essential for identifying cardiovascular disease (CVD) symptoms and distinguishing them from other health issues.
Initial Enquiry
Ask the patient about their primary concern and thoroughly explore all reported symptoms.
Medical History
Investigate past illnesses affecting the cardiovascular system, such as angina, anemia, rheumatic fever, congenital heart disease, stroke, thrombophlebitis, dysrhythmias, varicosities
Inquire about symptoms...
Initial Enquiry
Ask the patient about their primary concern and thoroughly explore all reported symptoms.
Medical History
Investigate past illnesses affecting the cardiovascular system, such as angina, anemia, rheumatic fever, congenital heart disease, stroke, thrombophlebitis, dysrhythmias, varicosities
Inquire about symptoms...
371


