使用大型预训练语言模型识别在线健康信息:混合方法研究研究
Dongmei Tan1, Yi Huang1, Ming Liu1
1College of Medical Informatics, Chongqing Medical University, Chongqing, China.
Journal of medical Internet research
|May 14, 2025
概括
大型语言模型 (LLM) 在识别在线健康错误信息方面表现有前途,ChatGPT-4显示出高准确度. 然而,对于专业的健康主题和细微的背景,LLM需要进一步改进.
科学领域:
- 人工智能的人工智能
- 自然语言处理自然语言处理.
- 医疗信息学 医疗信息学
背景情况:
- 在线健康信息很普遍,但往往含有不准确的信息,影响公共卫生决策.
- 误导性的健康声明给医疗保健系统带来了挑战.
- 大型语言模型 (LLM) 显示出识别健康错误信息的潜力,但其有效性尚未得到充分探索.
研究的目的:
- 评估四个主流LLM在识别在线健康信息方面的表现.
- 为LLMs在健康信息验证中的实际应用提供实证.
主要方法:
- 收集了2708个在线健康信息样本 (真实和错误的说法) 通过从谣言反驳网站的网络选.
- 利用LLM API进行真实性验证,使用专家结果作为基准.
- 通过语义相似性,准确性,回忆,F1得分,内容分析和可信度来评估模型性能.
主要成果:
- 这四个LLM都在识别在线健康信息方面表现良好.
- 聊天GPT-4获得了最高的准确度 (87.27%),紧随其后的是Ernie Bot (87.25%) 和iFLYTEK Spark (87%).
- 聊天GPT-4提供了最可靠的可信度评估,而Ernie Bot显示了与专家文本的最高语义相似性. 误判在与营养相关的主题中更为常见.
结论:
- 在线医疗信息识别 (LLM) 显示出在线医疗信息识别方面的潜力,但表现有差异.
- 模型擅长生成可访问的解释,但与专业的医学知识和新出现的健康问题作斗争.
- 进一步完善培训方法对于提高LLM可靠性,适应性和处理细微健康主题和文化背景的能力至关重要.
相关概念视频
Health Literacy
3.9K
Health literacy is an individual's or a community's capacity to comprehend, receive, read, and use relevant healthcare information and services. The World Health Organization (WHO, 2018) defines health literacy as the cognitive and social skills that determine the ability of individuals to gain access to, understand, and use information in ways that promote and maintain good health. As a result, the WHO helps individuals manage long-term health concerns, participate in preventative...
3.9K
Health Information Technology and Healthcare Information System
763
Health Information Technology (HIT)
Health Information Technology, commonly called HIT, integrates advanced information systems and technology in healthcare settings. Its primary functions include:
Health Information Technology, commonly called HIT, integrates advanced information systems and technology in healthcare settings. Its primary functions include:
763
Improving Translational Accuracy
2.5K
2.5K
Models of Health Promotion and Illness Prevention I
1.9K
A model is a theoretical way to understand a concept or an idea. Models can overcome barriers to health regardless of diverse economic and cultural backgrounds. In addition, models make the task easier by providing different ways to approach complex issues. There are two major health promotion models: the health belief model and the health promotion model.
The health belief model (HBM) attempts to predict health-related behavior in specific belief patterns. According to the HBM, a person's...
The health belief model (HBM) attempts to predict health-related behavior in specific belief patterns. According to the HBM, a person's...
1.9K
Models of Health Promotion and Illness Prevention II
1.5K
The person's health status fluctuates continually, varying from being in good health to becoming ill and returning to being healthy. To understand the concept of illness prevention, there are two models. First, the health-illness continuum model is a graphic representation of an individual's wellness. It states that a person is considered healthy in the absence of physical disease and the presence of good emotional health.
The agent-host-environment model states that disease results...
The agent-host-environment model states that disease results...
1.5K
Higher Mental Functions of the Brain: Language
640
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
640


