在大型语言中对文本质量的多标准评估 模型生成的妇产不良材料:DeepSeek对 OpenAI对克劳德
Tianying Zang1, Jiaojiao Li2, Lisha Wei2
1Department of Aesthetic Plastic Surgery and Laser Medicine, Beijing Anzhen Hospital Affiliated to Capital Medical University.
The Journal of craniofacial surgery
|September 10, 2025
概括
DeepSeek-R1和Claude-4-sonnet显示出对患者教育的承诺,DeepSeek-R1在内容质量方面表现出色,而Claude-4-sonnet则在妇痛信息的可读性方面表现出色.
科学领域:
- 人工智能在医学中的应用
- 医疗教育 技术 技术 医学教育
- 医疗信息学 医疗信息学
背景情况:
- 以患者为中心的医疗信息对于教育至关重要,需要可靠的在线资源.
- 评估人工智能产生的健康内容的质量和可读性对于不同患者的识字水平至关重要.
- 现有的数字健康信息质量各不相同,这对患者的理解构成了挑战.
研究的目的:
- 为了比较不同大型语言模型 (LLM) 的准确性和可读性,以获得妇产不良的医疗信息.
- 确定有效的AI工具,用于临床环境中的患者教育.
- 评估LLM产生的内容,以提高患者的理解和参与度.
主要方法:
- 从PubMed和谷歌趋势中选择了10个常见的妇产不良问题.
- 通过使用三个LLM生成响应:DeepSeek-R1,OpenAI-O3和Claude-4-Sonnet.
- 使用DISCERN-AI和PEMAT-AI评估文本质量;使用FKGL,FKRE,SMOG和ARI指标评估可读性.
主要成果:
- 深度搜索-R1获得了最高的整体内容质量评分 (DISCERN-AI,P=0.001).
- DeepSeek-R1显示了最高的平均单词和音节数量 (P<0.000).
- 克劳德-4-索内特显示出优异的可读性 (SMOG指数,P=0.028),而FKGL,FKRE和ARI显示没有显著的模型差异.
结论:
- 由于其质量和可读性,DeepSeek-R1和Claude-4-sonnet被推用于患者教育.
- 优先考虑这些模型可以提高患者的参与度,减少焦虑.
- 未来的研究应该通过整合这些优势,专注于开发更可靠的大规模医学LLM.
相关概念视频
Assessment of Airway, Skin Color, and Use of Accessory Muscles
1.6K
A thorough assessment of respiratory health is paramount in clinical settings to identify and manage respiratory distress and ensure adequate oxygenation. This article elaborates on the critical aspects of respiratory evaluation, including airway assessment, skin color examination, and the observation of accessory muscle use, which are integral to effectively diagnosing and managing patients with respiratory conditions.
Introduction
The initial evaluation of a patient's respiratory system...
Introduction
The initial evaluation of a patient's respiratory system...
1.6K
Multiple Comparison Tests
4.4K
Multiple comparison test, abbreviated as MCT, is a post hoc analysis generally performed after comparing multiple samples with one or more tests. An MCT will help identify a significantly different sample among multiple samples or a factor among multiple factors.
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
4.4K
Improving Translational Accuracy
3.6K
3.6K
Improving Translational Accuracy
14.1K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
14.1K
Goodness-of-Fit Test
8.1K
The goodness-of-fit test is a type of hypothesis test which determines whether the data "fits" a particular distribution. For example, one may suspect that some anonymous data may fit a binomial distribution. A chi-square test (meaning the distribution for the hypothesis test is chi-square) can be used to determine if there is a fit. The null and alternative hypotheses may be written in sentences or stated as equations or inequalities. The test statistic for a goodness-of-fit test is given as...
8.1K
Mesh Analysis
1.4K
Mesh analysis is a valuable method for simplifying circuit analysis using mesh currents as key circuit variables. Unlike nodal analysis, which focuses on determining unknown voltages, mesh analysis applies Kirchhoff's voltage law (KVL) to find unknown currents within a circuit. This method is particularly convenient in reducing the number of simultaneous equations that need to be solved.
A fundamental concept in mesh analysis is the definition of meshes and mesh currents. A mesh is a closed...
A fundamental concept in mesh analysis is the definition of meshes and mesh currents. A mesh is a closed...
1.4K
