多维评估大型语言模型对患者关于妊娠糖尿病问题的问题的反应
Betul Yigit Yalcın1, Ummu Mutlu2, Ayse Merve Ok3
1Department of Endocrinology, Faculty of Medicine, Istanbul University, Istanbul, Turkey. byigit667@gmail.com.
大型语言模型 (LLM) 在教育患者关于妊娠糖尿病 (GDM) 方面表现不同. 格洛克和双子座提供了更高质量的反应,但所有模型都需要改善患者的可读性.
科学领域:
- 医疗信息学 医疗信息学
- 医疗保健中的人工智能
- 患者教育 患者教育
背景情况:
- 孕期糖尿病 (GDM) 是常见的,需要明确的患者教育.
- 用于GDM患者教育的大型语言模型 (LLM) 的可靠性和可读性尚未得到充分证实.
- 准确的健康信息对于有效管理GDM至关重要.
研究的目的:
- 评估四个领先的LLM (ChatGPT-4o,Gemini 2.5 Pro,Grok 3.0,DeepSeek R-1) 在提供以患者为导向的GDM教育方面的表现.
- 评估LLM对临床GDM场景产生的响应的质量,可读性和词汇多样性.
- 确定在健康信息传递方面LLM改进的潜在领域.
主要方法:
- 利用从临床场景中获得的25个以患者为导向的GDM问题.
- 评估了四个LLM的响应:ChatGPT-4o,Gemini 2.5 Pro,Grok 3.0,以及DeepSeek R-1.这些 LLM的响应都得到了评估.
- 由七名内分泌学家使用修改后的DISCERN (mDISCERN) 和全球质量评分 (GQS) 评估反应质量.
- 分析了使用Flesch阅读方便 (FRES),Flesch-Kincaid等级水平 (FKGL),炮火雾指数 (GFI),科尔曼-利乌指数 (CLI) 和SMOG的可读性.
- 使用类型标记比率 (TTR) 测量词汇多样性.
主要成果:
- 格罗克和双子座获得了最高的专家评级质量评分 (mDISCERN和GQS).
- 与Grok和Gemini相比,ChatGPT-4o的质量显著降低 (p < 0.05).
- DeepSeek提供了最易读的内容,而Grok的回复是最长和最复杂的.
- 所有的LLM都制作了低于推的FRES值 (60) 的内容.
- 响应长度与质量呈正相关;词汇多样性与质量相反,但与可读性呈正相关.
结论:
- 在GDM患者教育的LLMs的表现中存在显著的变化.
- 在测试的模型中,Grok和Gemini在可靠的GDM信息传递方面显示出更高的潜力.
- 所有的LLM都需要提高可读性和内容复杂性,以满足患者理解标准.
- 特定模型的优化至关重要,以确保LLM为GDM等疾病提供安全有效的健康信息.
更多相关视频
09:00Author Spotlight: Validation of SICOLE-R for Assessing Cognitive and Reading Skills in Spanish-Speaking Children and Its Role in Personalized Education
Published on: August 16, 2024
07:22Glycemic Impact on Knee Osteoarthritis Symptoms on Physical, Radiographic, and Inflammatory Markers among Individuals Aged 50 and Over with Diabetes
Published on: March 7, 2025
相关概念视频
Diabetes Mellitus: Type 2 and Gestational
SBAR II: Application of SBAR
SBAR Report from a Nurse to a Health Care Provider
S: "Hello, Dr. Smith. This is Jane, RN, from the Med Surg unit. I am calling to tell you about Ms. White in Room 210, who is experiencing increased pain and redness at her incision site. Her recent...
Pathophysiology of Diabetes
Type 1 diabetes is characterized by autoimmune-mediated destruction of pancreatic β cells, with environmental factors potentially triggering this process in genetically susceptible individuals. Despite many not having a family history, certain genes increase susceptibility,...
Assessment of the Gastrointestinal System II: Health Perception Pattern
Health Perception Patterns
Health perception patterns offer valuable insights into a patient's lifestyle habits and how they may impact their GI health. These patterns include:
Diabetes Mellitus: Overview and Type I Subtype
Type 1 diabetes is an autoimmune disease in which the immune system mistakenly attacks and destroys the insulin-producing beta cells in the pancreas. As a result, the body is unable to produce sufficient insulin, and individuals with...
Oral Hypoglycemic Agents: Biguanides and Glitazones
