在USMLE STEP 2 CK和临床病例报告上评估ChatGPT 4.0的测试性能和临床诊断准确性
Allen Shieh1, Brandon Tran2, Gene He1
1Virginia Commonwealth University School of Medicine, Richmond, VA, USA.
Scientific reports
|April 23, 2024
概括
像ChatGPT 4.0这样的人工智能 (AI) 聊天机器人在回答美国医学执照考试 (USMLE) 问题和为临床病例生成差异诊断方面显示出更高的准确性.
科学领域:
- 医疗教育 技术 技术 医学教育
- 医疗保健中的人工智能
- 临床决策支持系统 临床决策支持系统
背景情况:
- 人工智能聊天机器人在包括医学在内的各种领域的使用越来越多.
- 有限的数据存在于AI的诊断准确性,特别是像ChatGPT 4.0这样的大型语言模型 (LLM),在临床环境中.
- 之前的AI聊天机器人版本在医学知识评估中表现不同.
研究的目的:
- 为了评估ChatGPT 4.0在回答美国医学执照考试 (USMLE) 第二步问题的准确性.
- 评估ChatGPT 4.0在从临床病例视频中生成准确差异诊断的能力.
- 将ChatGPT 4.0与其前身ChatGPT 3.5在标准化医疗问题上的性能进行比较.
主要方法:
- 109个USMLE第2步临床知识 (CK) 实践问题对ChatGPT 3.5和ChatGPT 4.0.0进行了管理.
- 聊天GPT 4.0提供了63个已发表的病例报告小组,以生成前三种差异诊断.
- 通过对其差异诊断进行排名来分析ChatGPT 4.0诊断准确性的信心.
主要成果:
- 在USMLE Step 2 CK问题上,ChatGPT 4.0显著提高了准确度,从47.7% (ChatGPT 3.5) 增加到87.2%.
- 在63个临床病例报告中,ChatGPT 4.0成功生成了74.6%的准确差异诊断.
- 在ChatGPT 4.0提供的正确诊断中,70.2%被列为最有可能的.
结论:
- 聊天GPT 4.0在回答标准化的医疗执照考试问题方面表现出提高的性能.
- 该研究强调了像ChatGPT 4.0这样的LLM作为临床病例分析和差异诊断生成工具的潜力.
- 人工智能技术的代改进正在提高其在医疗应用中的实用性和准确性.
相关概念视频
01:16Clinical Trials
Clinical trials are prospective experimental studies conducted on humans to determine the safety and efficacy of treatments, drugs, diet methods, and medical devices. Using statistics in clinical trials enables researchers to derive reasonable and accurate conclusions from the collected data, allowing them to make wise decisions in uncertain situations. In medical research, statistical methods are crucial for preventing errors and bias.
There are four phases in a clinical trial. A phase one...
There are four phases in a clinical trial. A phase one...
01:29Assessment of the Gastrointestinal System II: Health Perception Pattern
Assessing the gastrointestinal (GI) system is a complex process that begins with collecting subjective data. This data, collected through patient interviews, provides crucial insights into the patient's health history, perception patterns, and lifestyle habits, all contributing significantly to GI health.
Health Perception Patterns
Health perception patterns offer valuable insights into a patient's lifestyle habits and how they may impact their GI health. These patterns include:
Health Perception Patterns
Health perception patterns offer valuable insights into a patient's lifestyle habits and how they may impact their GI health. These patterns include:
01:26Nursing Assessment of the Genitourinary System II: Inspection and Palpation
The nursing assessment of the genitourinary (GU) system involves a systematic inspection and palpation to identify abnormalities in the kidneys, bladder, and surrounding structures.InspectionMouth: Inspect for signs of kidney dysfunction, such as stomatitis (inflammation of the mouth) and ammonia breath, which may occur in advanced kidney disease due to the buildup of urea, breaking down into ammonia.Skin: Check for pallor, which could indicate anemia caused by kidney disease. Look for...
01:30Urinary Tract Infection III: Diagnostic Studies and Interprofessional Care
A healthcare provider can diagnose a urinary tract infection (UTI) through several methods:Medical History and Symptoms: The provider will take a detailed medical history and ask about symptoms such as frequent urination, burning sensation during urination, and lower abdominal pain.Urinalysis: A clean-catch urine sample is collected in a sterile container and tested for the presence of bacteria, white blood cells (leukocytes), nitrites, blood, and protein. The presence of leukocytes and...
01:24Automated Microbial Diagnostics
Automated diagnostic analyzers have transformed clinical microbiology by providing rapid and reliable methods for pathogen identification and antibiotic susceptibility testing. Among these systems, the Vitek 2 is widely used because it automates the traditionally labor-intensive processes of microbial identification (ID) and antibiotic susceptibility testing (AST), delivering standardized and timely results that are essential for effective patient care.Microbial Identification with ID CardsThe...