放射学报告中错误检测的大型语言模型:封闭源和符合隐私的开源模型之间的比较分析
Babak Salam1,2, Claire Stüwe1, Sebastian Nowak1,2
1Department of Diagnostic and Interventional Radiology, University Hospital Bonn, Bonn, Germany.
European radiology
|February 20, 2025
概括
符合隐私的开源大语言模型 (LLM) 显示出在放射学报告中检测错误的前景,尽管封闭源模型提供了更高的准确性. 开源的LLMs为临床使用提供了一个可行的,机密的替代方案.
科学领域:
- 医疗成像中的人工智能
- 在医疗保健中的自然语言处理.
- 放射学报告 分析 分析
背景情况:
- 大型语言模型 (LLM) 可以帮助放射学报告的错误检测.
- 隐私问题限制了闭源LLMs的临床使用.
- 本研究评估了开源LLM作为一个符合隐私的替代方案.
研究的目的:
- 将开源LLM与闭源LLM在放射学报告中的错误检测性能进行比较.
- 评估符合隐私的LLM的临床适用性,以确保诊断报告的准确性.
主要方法:
- 分析了120份放射学报告 (X射线,超声波,CT,MRI).
- 在五个类别的100份报告中插入了397个错误.
- 评估了两个开源 (Llama 3-70b,Mixtral 8x22b) 和两个封闭源 (GPT-4,GPT-4o) 的LLM.
主要成果:
- 开源的LLM处理报告的速度更快 (6s ± 2s) 比闭源的LLM (13s ± 4s).
- 封闭源LLM实现了比开源模型 (Llama 3-70b: 79%,Mixtral 8x22b: 73%) 的更高的检测率 (GPT-4o: 88%,GPT-4: 83%).
- 数字错误比排版错误 (75%) 或解释错误 (70%) 更频繁地被发现 (88%).
结论:
- 开源的LLM提供了有效的,符合隐私的放射学报告错误检测.
- 虽然略低于封闭源代码模型的准确性,但开源LLM具有显著的临床潜力.
- 进一步开发开源的LLM可以通过可靠的报告来提高准确性和改善患者护理.
相关概念视频
Leaky Scanning
5.1K
During most eukaryotic translation processes, the small 40S ribosome subunit scans an mRNA from its 5' end until it encounters the first start AUG codon. The large 60S ribosomal subunit then joins the smaller one to initiate protein synthesis. The location of the translation initiation is largely determined by the nucleotides near the start codon as there may be multiple translation initiation sites present on the mRNA. Marilyn Kozak discovered that the sequence RCCAUGG (where R...
5.1K
Positron Emission Tomography
4.0K
Positron emission tomography (PET) is a medical imaging technique involving radiopharmaceuticals — substances that emit short-lived radiation. Although the first PET scanner was introduced in 1961, it took 15 more years before radiopharmaceuticals were combined with the technique and revolutionized its potential.
One of the main requirements of a PET scan is a positron-emitting radioisotope, which is produced in a cyclotron and then attached to a substance used by the part of the body...
One of the main requirements of a PET scan is a positron-emitting radioisotope, which is produced in a cyclotron and then attached to a substance used by the part of the body...
4.0K


