通过特定域的检索增强生成来增强大型语言模型:关于眼科长形式消费者健康问题的案例研究
Aidan Gilson1, Xuguang Ai2, Thilaka Arunachalam3
1Department of Ophthalmology and Visual Science, Yale School of Medicine, Yale University, New Haven, USA.
ArXiv
|October 1, 2025
概括
检索增强生成 (RAG) 通过减少幻觉证据显著改善了大型语言模型 (LLM) 的医疗问题答案. 然而,在最优的文档检索和选择方面仍然存在挑战,以提高准确性和完整性.
科学领域:
- 医疗信息学 医疗信息学
- 医疗保健中的人工智能
背景情况:
- 大型语言模型 (LLM) 在医学上表现有前途,但往往会产生没有支持的或幻觉的信息.
- 检索增强生成 (RAG) 是一种提高LLM事实性的技术,但其在专业医疗领域的应用需要彻底评估.
研究的目的:
- 开发和评估RAG管道,使用眼科专用文档来增强医学问答的LLM.
- 系统地评估RAG对LLM答案事实性,证据选择,归因,准确性和完整性的影响.
主要方法:
- 创建了一个RAG管道,包含大约7万份眼科文档.
- 10名医疗保健专业人员评估了LLM对100个长形式消费者健康问题的答案,并比较了RAG增强和非增强模型.
- 评估指标包括证据的真实性,排名,归因以及答案的准确性/完整性.
主要成果:
- 没有RAG的LLM产生了45.3%幻觉和34.1%错误引用的响应.
- RAG显著提高了响应准确度,达到54.5%,并减少了错误 (18.8%小幻觉,26.7%错误).
- RAG改善了证据归因,但在准确性和完整性方面略有下降;LLM并不总是使用排名最高的检索文件.
结论:
- RAG 显著提高了LLM生成的医疗信息的可靠性,减少了事实错误和幻觉.
- 在RAG的文档检索和选择过程中,仍然存在挑战,这影响了LLM在复杂任务中的表现,例如长形式的问答.
- 对特定领域的RAG和LLM技术的进一步进步是必要的,以克服目前证据利用和响应质量的局限性.
相关概念视频
01:25Glaucoma: Overview
1.2K
Glaucoma is an eye condition characterized by increased intraocular pressure that damages the retina and optic nerve, leading to irreversible blindness if left untreated. The human eye has various components, including the cornea, iris, pupil, lens, and optic nerve. Aqueous humor is secreted by the epithelium of the ciliary body in the posterior chamber and flows through the trabecular meshwork and canal of Schlemm, maintaining normal intraocular pressure. The trabecular meshwork and the canal...
1.2K
01:21Health Literacy
5.2K
Health literacy is an individual's or a community's capacity to comprehend, receive, read, and use relevant healthcare information and services. The World Health Organization (WHO, 2018) defines health literacy as the cognitive and social skills that determine the ability of individuals to gain access to, understand, and use information in ways that promote and maintain good health. As a result, the WHO helps individuals manage long-term health concerns, participate in preventative...
5.2K
01:29Documentation in Long-Term and Home Healthcare Setting
1.4K
Documentation in long-term care facilities and home healthcare settings is crucial for ensuring continuous, coordinated, and comprehensive care for patients. Each setting has its specific documentation processes and tools:
Long-Term Care Facilities
Long-Term Care Facilities
1.4K
01:45ER Retrieval Pathway
4.7K
In the secretory pathway, vesicles transport proteins from one cellular compartment to another in forward transport to deliver the protein to its correct location. Occasionally, misfolded proteins and incorrect proteins escape their original compartments, and a retrieval pathway is used to return the escaped proteins to their original compartment.
The ER uses many checkpoints to prevent the entry of incorrectly folded or a resident protein as cargo onto a transport vesicle. These mechanisms...
The ER uses many checkpoints to prevent the entry of incorrectly folded or a resident protein as cargo onto a transport vesicle. These mechanisms...
4.7K