评估三大语言模型在临床决策支持中的表现:基于实际病例的比较研究
Xueqi Wang1, Haiyan Ye1, Sumian Zhang1
1Department of Critical Care Medicine, Shanghai East Hospital, Tongji University School of Medicine, No.150, Jimo Road, Pudong New Area, Shanghai, China.
Journal of medical systems
|February 13, 2025
概括
在临床病例评估中,Med-Go的表现优于ChatGPT-4和Gemini,证明了大型语言模型 (LLM) 专业医疗培训的价值. 虽然LLM显示出潜力,但由于临床决策支持的不准确性,建议谨慎.
科学领域:
- 医疗信息学 医疗信息学
- 人工智能在医学中的应用
- 临床决策支持系统 临床决策支持系统
背景情况:
- 生成型大语言模型 (LLM) 在医疗保健中越来越多地被采用.
- 这些LLM的临床决策有效性需要彻底评估.
- 本研究调查了领先的LLM在现实世界医疗场景中的表现.
研究的目的:
- 评估和比较三个大型语言模型 (ChatGPT-4,Gemini,Med-Go) 在临床决策中的表现.
- 评估在各个学科LLM产生的医学建议的准确性和实用性.
- 确定当前LLM在处理复杂临床病例方面的优缺点.
主要方法:
- 从九个医学专业中利用了134个不同的临床病例.
- 每个LLM提供诊断,差异诊断,检查和治疗建议.
- 专家评审人员使用标准化标题对LLM的答案进行了评分.
主要成果:
- Med-Go获得了最高的中位数性能得分,超过了Gemini和ChatGPT-4 (p < 0.001).
- 根据任务,LLM的表现有所不同,差异诊断的结果较弱,治疗建议的结果较强.
- 在胃肠病学,脏病学和神经病学方面,Med-Go表现出了特殊的优势.
结论:
- 所有评估的LLM都超过了最高得分的60%,表明潜在的临床效用.
- 不准确性需要谨慎,强调需要仔细整合到临床工作流程中.
- 专业医疗培训,正如Med-Go所示,提高了LLM的表现,并为未来的发展提供了一条道路.
相关概念视频
Methods of Documentation VI: Case Management Model
559
The case management model is a multidisciplinary approach that involves healthcare professionals from diverse disciplines, such as physicians, nurses, therapists, social workers, and pharmacists, working collaboratively to address the various needs of patients. Each healthcare professional brings unique expertise and perspectives, contributing to a more comprehensive understanding of the patient's condition and tailoring treatment plans accordingly.
For example, a patient with a chronic...
For example, a patient with a chronic...
559
Nursing Clinical Information System
747
Nursing Clinical Information System (NCIS)
A Nursing Clinical Information System (NCIS) is a specialized type of healthcare information system tailored to meet the unique needs of nursing practice. It incorporates the principles of nursing informatics to streamline information management and improve the quality of care delivery.
Critical attributes of NCIS include:
A Nursing Clinical Information System (NCIS) is a specialized type of healthcare information system tailored to meet the unique needs of nursing practice. It incorporates the principles of nursing informatics to streamline information management and improve the quality of care delivery.
Critical attributes of NCIS include:
747
Clinical Trials
6.6K
Clinical trials are prospective experimental studies conducted on humans to determine the safety and efficacy of treatments, drugs, diet methods, and medical devices. Using statistics in clinical trials enables researchers to derive reasonable and accurate conclusions from the collected data, allowing them to make wise decisions in uncertain situations. In medical research, statistical methods are crucial for preventing errors and bias.
There are four phases in a clinical trial. A phase one...
There are four phases in a clinical trial. A phase one...
6.6K
Documentation of Nursing Diagnosis
1.2K
The nurse documents nursing diagnoses and enters them into the patient record. The identified patient's nursing diagnosis is either written out with a plan of care or entered into the electronic health record.
In some settings, data-driven computerized decision support systems are in place, allowing for more accurate nursing diagnoses. The database within one of these systems includes diagnostic labels defining characteristics, activities, and indicators for nursing. A nurse enters...
In some settings, data-driven computerized decision support systems are in place, allowing for more accurate nursing diagnoses. The database within one of these systems includes diagnostic labels defining characteristics, activities, and indicators for nursing. A nurse enters...
1.2K
Cancer Survival Analysis
321
Cancer survival analysis focuses on quantifying and interpreting the time from a key starting point, such as diagnosis or the initiation of treatment, to a specific endpoint, such as remission or death. This analysis provides critical insights into treatment effectiveness and factors that influence patient outcomes, helping to shape clinical decisions and guide prognostic evaluations. A cornerstone of oncology research, survival analysis tackles the challenges of skewed, non-normally...
321
Patient-centered Care
1.9K
Patient-centered care involves delivering care beyond inpatient hospitalization. Reflective practice can enhance a patient-centered approach. Reflective practice is a process of reasoning that considers all aspects of the present situation, including practicalities, learning from personal practice, and consideration of patient needs. Patients appreciate care decisions made while considering their input. Involving the patient in their care provides the patient with a sense of contribution rather...
1.9K


