评估大型语言模型作为医学学习者和临床医生使用先进提示技术的诊断工具
Karolina Gaebe1, Benjamin van der Woerd1
1Division of Otolaryngology-Head and Neck Surgery, Department of Surgery - Michael G. DeGroote School of Medicine, Hamilton, Ontario, Canada.
PloS one
|August 1, 2025
概括
开源大型语言模型 (LLM) 作为诊断工具具有前景,在使用高级提示函数工程时,其性能优于GPT-3.5等专有模型. 这项研究强调了开源LLM对于可复制医疗AI开发的价值.
科学领域:
- 人工智能在医学中的应用
- 自然语言处理自然语言处理.
- 临床决策支持 临床决策支持
背景情况:
- 大型语言模型 (LLM) 在医疗诊断方面显示出潜力.
- 之前的研究主要使用专有LLM,忽视了先进的提示工程.
- 开源的LLM提供了研究和开发的替代方案.
研究的目的:
- 评估三个开源LLM的诊断准确性.
- 评估高级快速工程对LLM绩效的影响.
- 将开源LLM的性能与GPT-3.5.5等专有模型进行比较.
主要方法:
- 测试了三个开源的LLM (拉玛-3.1-70b-多功能,拉玛-3.1-8b-即时,mixtral-8x7b-32768) 的结果.
- 应用先进的快速工程技术应用于临床挑战问题.
- 绩效指标包括正确性,准确性,精度,敏感性和特异性.
- 结果与已公布的GPT-3.5数据进行比较,对基本提示和视觉资产排除的灵敏度分析.
主要成果:
- 开源的LLM实现了比GPT-3.5 (74%) 的更高的正确答案率 (79%,65%,62%).
- 诊断准确度,精度,灵敏度和特异性也超过了GPT-3.5.5.
- 与基本提示相比,高级提示显著提高了LLM性能.
- 排除视觉资产并没有改变这些业绩趋势.
结论:
- 先进的提示工程使开源LLM能够产生临床准确的诊断反应.
- 专有模型在可访问性和可重复性方面面临限制.
- 未来的研究应该优先考虑开源的LLM,并为医疗AI的可复制性提供快速工程.
相关概念视频
Modeling in Therapy
149
Modeling, a key technique in therapy, uses observational learning to help clients acquire and practice new skills by watching therapists demonstrate desired behaviors. This approach, rooted in Albert Bandura's concept of vicarious learning, plays a significant role in therapeutic interventions for various psychological conditions, including social anxiety, ADHD, and depression.
Participant Modeling
Participant modeling involves therapists demonstrating calm and effective behaviors in...
Participant Modeling
Participant modeling involves therapists demonstrating calm and effective behaviors in...
149
Language and Cognition
444
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
444
Techniques of Therapeutic Communication II: Focusing, Paraphrasing, and Summarizing
9.1K
Focusing involves centering a conversation on a message's critical elements or concepts. Focusing is valuable if the talk is vague or patients begin to repeat themselves. Sometimes, when patients are asked about their symptoms, they may go off-topic and try to tell their entire life story. Respectfully, the nurse should bring the conversation back into focus.
This therapeutic technique can also be used when a patient brings up pertinent information during a health-related conversation. The...
This therapeutic technique can also be used when a patient brings up pertinent information during a health-related conversation. The...
9.1K


