开发和评估基于证据的患者教育的基于LLM的RAG框架
AlHasan AlSammarraie1, Ali Al-Saifi2, Hassan Kamhia3
1Hamad Bin Khalifa University College of Science and Engineering, Doha, Qatar aalsammarraie@hbku.edu.qa.
BMJ health & care informatics
|July 25, 2025
概括
代理检索增强生成 (ARAG) 改进了使用大型语言模型 (LLM) 的阿拉伯患者教育材料 (PEM) 生成. 较大的模型对于有效验证和阻止有害内容至关重要.
科学领域:
- 人工智能的人工智能
- 自然语言处理自然语言处理.
- 医疗信息学 医疗信息学
背景情况:
- 开发高质量的,基于证据的患者教育材料 (PEMs) 对于有效的医疗保健沟通至关重要.
- 大型语言模型 (LLM) 提供了自动化内容生成的潜力,但需要仔细评估准确性和安全性,特别是在阿拉伯语等专业语言中.
研究的目的:
- 开发和评估一种代理检索增强生成 (ARAG) 框架,用于使用开源LLMs生成阿拉伯PEM.
- 评估LLM作为验证代理来阻止生成PEM中的有害内容的能力.
主要方法:
- 在四个设置中测试了12个LLM:基础,基础与快速工程,ARAG和ARAG与快速工程.
- 使用自动化LLM评估和专家审查,在五个指标上评估PEM质量:准确性,可读性,全面性,适当性和安全性.
- 使用有害/安全PEM数据集来评估验证剂的性能,以测量阻断的准确性.
主要成果:
- 在12个LLM中,ARAG设置提高了10个LLM的生成性能,以阿拉伯语为中心的模型主导了顶级排名.
- 使用ARAG和快速工程的AceGPT-v2-32B展示了最高的性能.
- 验证剂的准确性与模型大小有很强的相关性 (需要≥27B参数才能达到>0.80准确性),尽管Fanar-7B在生成方面表现出色,但在验证方面没有表现出色.
结论:
- 以阿拉伯语为中心的LLMs显示了阿拉伯语PEM生成的优势.
- ARAG框架显著提高了PEM生成质量,尽管上下文限制影响了大背景模型.
- 模型大小是LLMs可靠的有害内容验证的关键因素.
更多相关视频
相关概念视频
01:20Nursing Process for Patient and Caregiver Teaching III: Evaluation and Documentation
1.9K
Evaluation of the teaching process enables the nurse to determine if the patient's learning needs were met and if training was effective. If the expected outcomes are not met, the care plan is revised, and additional education or reinforcement is provided. Nurses can ask questions after the session or obtain feedback to assess the patient's understanding of the topic.
Nurses can use several methods to evaluate patient outcomes. For example, oral questions can assess cognitive learning,...
Nurses can use several methods to evaluate patient outcomes. For example, oral questions can assess cognitive learning,...
1.9K
01:11Guidelines for Writing Outcome
3.0K
When developing expected outcomes for a patient care plan, the nurse should adhere to the following recommendations:
Patient outcomes reflect the patient's response to the goal rather than what the nurse aims to achieve. Terminology should be observable and measurable to avoid the reader's interpretation. The desired outcome should be realistic and achievable in the designated care timeframe. Expected outcomes should align with adjunctive therapies. The outcome should enhance care...
Patient outcomes reflect the patient's response to the goal rather than what the nurse aims to achieve. Terminology should be observable and measurable to avoid the reader's interpretation. The desired outcome should be realistic and achievable in the designated care timeframe. Expected outcomes should align with adjunctive therapies. The outcome should enhance care...
3.0K
01:21Health Literacy
4.2K
Health literacy is an individual's or a community's capacity to comprehend, receive, read, and use relevant healthcare information and services. The World Health Organization (WHO, 2018) defines health literacy as the cognitive and social skills that determine the ability of individuals to gain access to, understand, and use information in ways that promote and maintain good health. As a result, the WHO helps individuals manage long-term health concerns, participate in preventative...
4.2K
01:08Role of Communication in the Nursing Process III: Evaluation and Documentation
1.4K
A successful patient outcome depends mainly on the evaluation stage of the nursing process. Evaluation determines effectiveness by reviewing what was done previously after the completion of nursing interventions. Every time a healthcare professional steps in or administers treatment, they must reassess or evaluate the action to ensure the intended result. During the evaluation phase, there are three probable patient outcomes:
1.4K
01:24Nursing Process for Patient and Caregiver Teaching I: Assessment and Diagnosis
1.7K
The nursing process provides a clinical decision-making framework for patients and families to establish and implement a personalized care plan. Since part of the nurse's duties is to teach patients, the steps of the nursing process are the most effective way to approach instruction. The nursing process and the teaching-learning process are inextricably linked.
It is critical to determine the patient's learning needs during the assessment. Determination of learning needs compounds data...
It is critical to determine the patient's learning needs during the assessment. Determination of learning needs compounds data...
1.7K
01:30Methods of Documentation VII: EMR
914
Electronic Medical Records (EMRs) primarily center around electronically documenting patients' health information within a single healthcare organization or practice. They contain essential clinical data related to a patient's medical history, diagnoses, medications, treatment plans, lab results, and other pertinent information relevant to the specific encounter or episode of care. EMRs are designed to streamline documentation and workflow processes within individual healthcare...
914