大型语言模型的定量评估,以简化放射学报告的印象:多模式的回顾分析
Rushabh Doshi1, Kanhai S Amin1, Pavan Khosla1
1From the Yale School of Medicine (R.D., P.K.) and Department of Radiology and Biomedical Imaging (K.S.A., S.S.B., S.C., H.P.F.), Yale School of Medicine, 333 Cedar St, New Haven, CT 06510; Yale School of Management, New Haven, Conn (H.P.F.); and Department of Health Policy and Management, Yale School of Public Health, New Haven, Conn (H.P.F.).
Radiology
|March 26, 2024
概括
大型语言模型 (LLM) 可以简化患者的复杂放射学报告. 四个经过测试的LLM成功地减少了各种提示的报告可读性,使医疗信息更容易获得.
科学领域:
- 医疗信息学 医疗信息学
- 医疗保健中的人工智能
- 放射学通信 放射学通信
背景情况:
- 放射学报告中的复杂医学术语可能会导致患者的困惑和焦虑.
- 增加患者对电子健康记录的访问,需要提高报告可读性.
- 大型语言模型 (LLM) 为简化临床文档提供了一个潜在的解决方案.
研究的目的:
- 评估四种公开可用的LLM在简化放射学报告印象方面的有效性.
- 为了比较ChatGPT-3.5,ChatGPT-4,Gemini (以前Bard) 和Bing在生成患者友好的放射学摘要方面的性能.
- 评估不同的提示策略如何影响LLM简化能力.
主要方法:
- 从MIMIC-IV数据库中对750个匿名放射学报告印象进行了回顾性分析.
- 利用三个不同的提示来测试LLM简化:一般简化,患者特定简化和七年级阅读水平简化.
- 通过从四个已确定的索引中得出的可读性得分来评估简化,并将结果与Wilcoxon签名级别测试进行比较.
主要成果:
- 所有四个LLM都表现出了统计学上显著的能力,可以简化所有测试提示的放射学报告印象 (P < .001).
- 将受众指定为患者或要求七年级阅读水平进一步降低了所有模型的简化报告的复杂性.
- 根据使用的提示符,在LLM中观察到简化性能的变化.
结论:
- 所有评估的LLM都成功地简化了放射学报告的印象,表明它们在改善患者理解方面具有潜在的实用性.
- 提示性措辞显著影响了LLM所取得的简化程度.
- 在提高患者对不同成像方式和解剖区域的放射学发现的理解方面,LLM是有前途的.
相关概念视频
02:00Molecular Models
Physical models representing molecular architectures of chemical compounds play essential roles in understanding chemistry. The use of molecular models makes it easier to visualize the structures and shapes of atoms and molecules.
01:07What is a Mode?
The mode is one of the commonly used measures of a central tendency. It is defined as the most frequent value in a data set.
There can be more than one mode in a data set if multiple values have the same highest frequency. For instance, suppose that the Statistics exam scores of 20 students are: 50; 53; 59; 59; 63; 63; 72; 72; 72; 72; 72; 76; 78; 81; 83; 84; 84; 84; 90; 93. Here, the mode is 72, as it occurs most frequently, five times.
A data set with two modes is called bimodal. For example,...
There can be more than one mode in a data set if multiple values have the same highest frequency. For instance, suppose that the Statistics exam scores of 20 students are: 50; 53; 59; 59; 63; 63; 72; 72; 72; 72; 72; 76; 78; 81; 83; 84; 84; 84; 90; 93. Here, the mode is 72, as it occurs most frequently, five times.
A data set with two modes is called bimodal. For example,...
01:23Mechanistic Models: Compartment Models in Individual and Population Analysis
Mechanistic models are utilized in individual analysis using single-source data, but imperfections arise due to data collection errors, preventing perfect prediction of observed data. The mathematical equation involves known values (Xi), observed concentrations (Ci), measurement errors (εi), model parameters (ϕj), and the related function (ƒi) for i number of values. Different least-squares metrics quantify differences between predicted and observed values. The ordinary least squares (OLS)...
01:29Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
01:12Modeling and Similitude
Scaled modeling is a fundamental technique in engineering, enabling the study of large and complex systems by creating smaller, manageable replicas that recreate critical characteristics of the original. In hydrology and civil infrastructure, for example, scaled models of dams help analyze water flow, turbulence, and pressure. This method allows for accurate predictions of real-world behavior within a controlled environment, significantly reducing the cost and time involved in full-scale...
01:30Typical Model Studies
Fluid mechanics model studies often utilize scaled-down systems to predict fluid behavior in full-scale environments, such as river flows, dam spillways, and structures interacting with open surfaces. Maintaining Froude number similarity in river models is crucial, as it replicates surface flow features like wave patterns and velocities.