利用大型语言模型改善患者访问和自我管理:评估员对专家和人工智能生成内容的盲目比较
Xiaolei Lv1,2,3,4,5,6, Xiaomeng Zhang1,2,3,4,5,6, Yuan Li1,2,3,4,5,6
1Department of Oral and Maxillofacial Implantology, Shanghai PerioImplant Innovation Center, Shanghai Ninth People's Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China.
Journal of medical Internet research
|April 25, 2024
概括
大型语言模型 (LLM) 在口腔健康方面表现有前途. 聊天GPT-4在提供以患者为中心的口腔健康建议方面表现出高的稳定性和可靠性,与人类专家相比.
科学领域:
- 医疗保健中的人工智能
- 口腔健康信息学 口腔健康信息学
- 数字健康数字健康
背景情况:
- 大型语言模型 (LLM) 在各个领域都显示出潜力.
- 需要对专业医疗保健,特别是口腔健康的LLM进行全面评估.
- 评估LLM对患者护理获取和质量的影响至关重要.
研究的目的:
- 为了评估谷歌Bard,ChatGPT-3.5和ChatGPT-4的有效性.
- 将常见口腔健康问题的LLM建议与人类牙科专家进行比较.
- 在模拟的临床场景中评估LLM绩效.
主要方法:
- 一个比较分析涉及40个患者衍生的口腔疾病问题.
- 人类专家和LLM的回复被牙医和普通用户评估.
- 评估标准包括可读性,适当性,无害性,全面性,捕捉意图和有用性.
- 通过重复的试验来评估LLM响应稳定性.
主要成果:
- 谷歌Bard在可读性方面表现出色,但不如人类专家那么合适 (P=.03).
- 聊天GPT-3.5和聊天GPT-4对人类专家显示出相似的适用性.
- 聊天GPT-4表现出最高的稳定性和可靠性.
- 所有的LLM在无害性方面得分很高,在帮助性和普通用户捕捉意图方面感知到的差异很小.
结论:
- 在口腔健康方面,LLM,特别是ChatGPT-4,为患者教育和临床护理提供了潜力.
- 绩效差异突出了LLM持续改进的必要性和医疗保健中的伦理考虑.
- 未来的研究将重点关注临床环境中LLMs的安全整合策略.
相关概念视频
01:16Non-equilibrium in the Cell
4.4K
An important concept in studying metabolism and energy is that of chemical equilibrium. Most chemical reactions are reversible. They can proceed in both directions, releasing energy into their environment in one direction, and absorbing it from the environment in the other direction. The same is true for the chemical reactions involved in cell metabolism, such as the breaking down and building up of proteins into and from individual amino acids, respectively. Reactants within a closed system...
4.4K
01:13Patient-centered Care
2.0K
Patient-centered care involves delivering care beyond inpatient hospitalization. Reflective practice can enhance a patient-centered approach. Reflective practice is a process of reasoning that considers all aspects of the present situation, including practicalities, learning from personal practice, and consideration of patient needs. Patients appreciate care decisions made while considering their input. Involving the patient in their care provides the patient with a sense of contribution rather...
2.0K
01:30Health Information Technology and Healthcare Information System
834
Health Information Technology (HIT)
Health Information Technology, commonly called HIT, integrates advanced information systems and technology in healthcare settings. Its primary functions include:
Health Information Technology, commonly called HIT, integrates advanced information systems and technology in healthcare settings. Its primary functions include:
834
02:16Stereotype Content Model
14.7K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
14.7K
01:15Methods of Documentation VI: Case Management Model
570
The case management model is a multidisciplinary approach that involves healthcare professionals from diverse disciplines, such as physicians, nurses, therapists, social workers, and pharmacists, working collaboratively to address the various needs of patients. Each healthcare professional brings unique expertise and perspectives, contributing to a more comprehensive understanding of the patient's condition and tailoring treatment plans accordingly.
For example, a patient with a chronic...
For example, a patient with a chronic...
570
01:21Health Literacy
4.0K
Health literacy is an individual's or a community's capacity to comprehend, receive, read, and use relevant healthcare information and services. The World Health Organization (WHO, 2018) defines health literacy as the cognitive and social skills that determine the ability of individuals to gain access to, understand, and use information in ways that promote and maintain good health. As a result, the WHO helps individuals manage long-term health concerns, participate in preventative...
4.0K