使用大型语言模型评估YouTube上的医疗内容的质量
Mahmoud Khalil1, Fatma Mohamed2, Abdulhadi Shoufan3,4
1Computer Science Department, Western University, London, ON, Canada.
Scientific reports
|March 23, 2025
概括
大型语言模型 (LLM) 可以评估YouTube上的医疗视频质量,显示与专家评级的不同一致. 提供评分准则提高了LLM在评估健康信息准确性的表现.
科学领域:
- 数字健康数字健康
- 人工智能在医学中的应用
- 卫生沟通健康沟通
背景情况:
- YouTube是健康信息的主要来源,但内容准确性是一个重大问题.
- 专家对所有医疗视频的审查是不切实际的,因为每天上传的数量很大.
- 评估在线健康信息的质量对于公共卫生和知情决策至关重要.
研究的目的:
- 探索大型语言模型 (LLM) 在评估YouTube上的医疗内容质量的潜力.
- 使用DISCERN工具,将基于LLM的质量评估与专家评估进行比较.
- 调查影响医学视频质量评估LLM绩效的因素.
主要方法:
- 收集了一组YouTube医疗视频的数据集,这些视频以前是由人类专家评分的.
- 促使20名LLM使用DISCERN仪器对视频质量进行评分.
- 使用布伦南-普雷迪格 (BP) 的卡帕统计数据分析了LLM和专家评级之间的互评级可靠性.
主要成果:
- 专家评级对LLM的认同范围很广,从-1.10到0.82.
- 观察到LLM的总体趋势是比人类专家赋予更高的分数.
- 个人问题上的一致性较低,在特定标准上的LLM和专家判断之间存在明显的差异.
- 将评分准则纳入提示,提高了LLM的绩效.
结论:
- 一些LLM在评估YouTube上的医疗视频质量方面表现出能力.
- 实际上,LLM可以作为独立的工具,也可以整合到推系统中,以解决在线不可靠的健康信息问题.
- 需要进一步的研究来完善LLM的准确性,并解决健康信息评估中的偏见.
相关概念视频
02:07Improving Translational Accuracy
2.5K
2.5K
01:21Health Literacy
3.9K
Health literacy is an individual's or a community's capacity to comprehend, receive, read, and use relevant healthcare information and services. The World Health Organization (WHO, 2018) defines health literacy as the cognitive and social skills that determine the ability of individuals to gain access to, understand, and use information in ways that promote and maintain good health. As a result, the WHO helps individuals manage long-term health concerns, participate in preventative...
3.9K
01:13Patient-centered Care
1.9K
Patient-centered care involves delivering care beyond inpatient hospitalization. Reflective practice can enhance a patient-centered approach. Reflective practice is a process of reasoning that considers all aspects of the present situation, including practicalities, learning from personal practice, and consideration of patient needs. Patients appreciate care decisions made while considering their input. Involving the patient in their care provides the patient with a sense of contribution rather...
1.9K
01:15Nursing Evaluation
3.2K
The evaluation stage signals the end of the nursing process. The nurse gathers evaluative data to assess whether or not the patient has attained the expected results. Whereas the nurse collects data in the nursing assessment to identify the patient's health concerns, the evaluation stage data determines if the indicated health issues are resolved. Evaluative data collection includes two sections: the data acquired to evaluate patient outcomes and the time criteria for data collection.
3.2K
01:17Classification of Illness
7.2K
The meaning of illness is individualized to each person who experiences an alteration in health. In contrast, disease is a medical term indicating a pathological change in the structure and function of the body or mind. It is a condition that has specific symptoms and boundaries.
An illness is a response to a disease in which the person's level of functioning is changed compared with a previous level. The general classification of illness includes acute and chronic.
Acute illness is severe...
An illness is a response to a disease in which the person's level of functioning is changed compared with a previous level. The general classification of illness includes acute and chronic.
Acute illness is severe...
7.2K
02:51mRNA Stability and Gene Expression
2.8K
2.8K