病理学中的大型语言模型:对聊天GPT和巴德的比较研究,病理学实习生对多项选择题进行了比较
Wei Du1, Xueting Jin1, Jaryse Carol Harris1
1Department of Pathology and Laboratory Medicine, Hospital of the University of Pennsylvania, Philadelphia, PA, United States of America.
Annals of diagnostic pathology
|November 8, 2024
概括
大型语言模型 (LLM) 在病理学中表现有前途. 在多选题问题上,ChatGPT显著优于Bard和实习生,显示出高准确性和一致性.
科学领域:
- 人工智能在医学中的应用
- 数字病理学数字病理学
- 医疗教育 技术 技术 医学教育
背景情况:
- 大型语言模型 (LLM) 在各种医学领域都显示出潜力.
- 评估LLM在病理学等专业领域的表现对于理解它们的实用性至关重要.
研究的目的:
- 用多选题评估ChatGPT和Bard在病理学中的表现.
- 为了比较LLM的表现与病理学学员 (初级和高级) 的表现.
- 分析在多个测试会话中LLM响应的一致性和可变性.
主要方法:
- 使用了来自15个病理学子专业 (无图像) 的150个多选题.
- 在2023年6月至2024年1月期间,ChatGPT和Bard在三个会议中进行了测试.
- 答案与16名病理学实习生进行了比较;根据实习生的表现,问题根据难度分类.
主要成果:
- 聊天GPT的平均得分为82.2%,明显优于巴德 (49.5%) 和实习生 (初级:45.1%,高级:56.0%).
- 聊天GPT在难题 (63.4%-68.3%) 中表现出色,而在易题 (73.7%-100.0%) 中,实习生得分高.
- 聊天GPT显示出高的一致性 (80%-85%),而巴德显示出更大的变异性 (54%-61%).
结论:
- 法律学士,特别是ChatGPT,显示出病理学教育和实践的巨大潜力.
- 聊天GPT的卓越性能和一致性凸显了它在这个领域的先进能力.
- 持续的开发和基本的人类监督是必要的,以确保LLMs在病理学中的可靠临床整合.
相关概念视频
Introduction to Language of Pathophysiology l
Pathophysiology investigates how biological mechanisms—typically starting at the cellular level—disrupt normal bodily functions. It bridges anatomy and physiology to explain the progression of disease. With this foundation, it is important to understand the following key terms used to describe disease processes: Diagnosis:The process of identifying a disease using clinical evaluation, including signs (objective evidence like rashes), symptoms (subjective experiences like pain), laboratory test...
Introduction to Language of Pathophysiology ll
This lesson explores key terms that describe how diseases progress, their outcomes, and their distribution in populations.Diagnostic tests identify diseases and monitor treatment. These include blood and urine tests, biopsies, imaging (X-ray, MRI), and detection of infectious agents.Remission is a reduction or disappearance of symptoms.Exacerbation refers to the worsening of symptoms, such as increased wheezing during an asthma attack.A precipitating factor triggers an acute episode, while a...


