定制的大型语言模型提高了准确性:将检索增强生成和人工智能代理与基于证据的医学非定制模型进行比较
Joshua J Woo1, Andrew J Yang1, Reena J Olsen2
1Brown University/The Warren Alpert School of Brown University, Providence, Rhode Island, U.S.A.
使用Retrieval Augmented Generation (RAG) 和AI代理来定制大型语言模型 (LLM) 显著提高了医疗信息的准确性,在骨科病例中表现优于标准LLM.
科学领域:
- 医疗信息学 医疗信息学
- 人工智能在医学中的应用
- 整形外科手术 整形外科手术
背景情况:
- 大型语言模型 (LLM) 在医学上表现有前途,但由于可变的准确性而面临怀疑.
- 标准的LLM可能不会始终提供可靠的医疗信息.
- 需要定制技术来提高临床应用的LLM准确性.
研究的目的:
- 评估检索增强生成 (RAG) 和代理增强在提高LLM准确性的有效性.
- 将定制的LLM方法与使用前十字带 (ACL) 损伤病例的标准LLM进行比较.
- 展示定制的LLM在提供准确的医疗信息方面的价值.
主要方法:
- 根据2022年AAOS指南策划了100个ACL相关问题和答案.
- 测试了标准的LLM (闭源和开源),然后用RAG和AI代理增强.
- 经过奖学金培训的外科医生盲目评估了响应的准确性;ROUGE和MEAT得分被计算出来.
主要成果:
- 非定制的LLM执行精度低于60%.
- 在所有模型中,RAG的精度平均提高了39.7%.
- 梅塔的Llama3 70b (仅用于RAG) 达到了94%的准确率;OpenAI的GPT-4 (RAG + AI代理) 达到了95%.
结论:
- 检索增强生成 (RAG) 在医疗环境中显著提高了LLM准确性.
- 代理增强进一步完善了LLM性能,实现了近乎完美的准确性.
- 定制的LLM,如RAG和代理增强模型,可以作为骨科信息的可靠来源,支持患者的决策.
更多相关视频
09:35A Protocol for Using Gene Set Enrichment Analysis to Identify the Appropriate Animal Model for Translational Research
Published on: August 16, 2017
09:37Navigating MARRVEL, a Web-Based Tool that Integrates Human Genomics and Model Organism Genetics Information
Published on: August 15, 2019
相关概念视频
Combination Therapies and Personalized Medicine
The combination of the drug acetazolamide and sulforaphane is a good example of combination therapy to treat cancer. The cells in the interior of a large tumor often die due to the hypoxic and...
Mouse Models of Cancer Study
The development of transgenic, knockout, and knock-in mice has led to an exponential increase in their use as model organisms in research,...
Methods of Documentation VI: Case Management Model
For example, a patient with a chronic...
Models of Health Promotion and Illness Prevention I
The health belief model (HBM) attempts to predict health-related behavior in specific belief patterns. According to the HBM, a person's...
Genomics
Evolutionary Relationships through Genome Comparisons
