对更年期和激素治疗的大型语言模型响应的准确性和可读性的评估
Jana Karam1,2, Chrisandra Shufelt1,2,3, Nancy Safwan1,2
1Division of General Internal Medicine, Mayo Clinic, Jacksonville, FL.
概括
大型语言模型 (LLM) 对更年期和激素治疗问题的准确性有限. 需要进一步开发以确保为患者和临床医生提供可靠的健康信息.
科学领域:
- 医疗保健人工智能的人工智能
- 医疗信息学 医疗信息学
- 自然语言处理自然语言处理.
背景情况:
- 生成型人工智能 (AI) 越来越多地被用于医疗保健教育.
- 大型语言模型 (LLM) 为患者和临床医生提供支持的潜力.
- 准确可靠的健康信息对于有效的患者护理和教育至关重要.
研究的目的:
- 为了评估四个LLM (ChatGPT 3.5,Gemini,ChatGPT 4.0,OpenEvidence) 的准确性,完整性和可读性.
- 评估LLM在回答患者层面和临床医生层面关于更年期和激素治疗的问题上的表现.
- 确定LLM产生的健康内容需要改进的领域.
主要方法:
- 35个问题 (20个患者层面,15个临床医生层面) 被提出给四个LLMs.
- 专家审查员评估了答案的准确性和完整性.
- 用弗莱什阅读易度得分 (FRES) 测量了患者级响应可读性.
- 统计分析包括ANOVA和赔率比率.
主要成果:
- 对于患者问题,ChatGPT 3.5的准确度最高 (70%),而Gemini的准确度明显低 (30%).
- 对患者问题的LLM答案通常难以读取 (FRES分数低).
- 对于临床医生的问题,ChatGPT 4.0是最准确的 (67%),其他模型之间没有显著差异.
结论:
- 在回答更年期相关的健康问题时,LLM的准确性和完整性有限.
- 目前的LLM需要显著改进,以便在患者和临床医生教育中可靠使用.
- 在医疗保健中广泛采用LLM之前,确保准确性和可靠性至关重要.
更多相关视频
06:18An In Vivo Estrogen Deficiency Mouse Model for Screening Exogenous Estrogen Treatments of Cardiovascular Dysfunction After Menopause
Published on: August 13, 2019
12.8K
14:26A Method to Study the Impact of Chemically-induced Ovarian Failure on Exercise Capacity and Cardiac Adaptation in Mice
Published on: April 7, 2014
16.0K
相关概念视频
Menopause
3.5K
Menopause, a natural biological process marking the end of a woman's fertility, typically occurs between the fifth and sixth decade of life. This phase is characterized by the exhaustion of the ovarian follicle pool, leading to less responsive ovaries despite the high levels of Follicle Stimulating Hormone (FSH) and Luteinizing Hormone (LH). The consequential decrease in estrogen production results in symptoms like hot flashes, heavy sweating, headaches, hair loss, muscle pains, vaginal...
3.5K
Hormonal Regulation
35.6K
The renin-aldosterone system is an endocrine system which guides the renal absorption of water and electrolytes, thus managing blood pressure and osmoregulation. Activation of the system begins in the kidneys with a small cluster of cells adjacent to the afferent and efferent blood vessels of the renal corpuscle. As the nephrons are filtering blood, juxtaglomerular cells monitor blood pressure. If they detect a decrease in pressure, they release the hormone renin into the bloodstream.
35.6K
Hormonal Regulation
47.7K
Hormones regulate a significant portion of digestion through activation of the neuroendocrine system. The neuroendocrine system of digestion contains many different hormones all with multiple functions that are both, directly and indirectly, involved in digestion.
47.7K
Target Cell Response to Hormones
5.1K
Hormones intricately bind to receptors on the surface or within target cells, initiating a cascade of cellular responses.
Notably, the cellular response can be regulated by altering the number of receptors expressed in the cell. For example, prolonged exposure to elevated hormone levels results in a gradual decline or down-regulation in the number of receptors for that specific hormone on the cell surface. Conversely, in response to low hormone levels, cells may use up-regulation, producing an...
Notably, the cellular response can be regulated by altering the number of receptors expressed in the cell. For example, prolonged exposure to elevated hormone levels results in a gradual decline or down-regulation in the number of receptors for that specific hormone on the cell surface. Conversely, in response to low hormone levels, cells may use up-regulation, producing an...
5.1K
Hormonal Regulation of Blood Pressure
5.8K
Endocrinal or hormonal intervention in the cardiovascular system is predominantly exerted by the catecholamines - epinephrine and norepinephrine, as well as a slew of hormones that interact with renal function to modulate blood volume.
Epinephrine and Norepinephrine
The adrenal medulla releases epinephrine and norepinephrine, catecholamines that enhance and extend the sympathetic or "fight or flight" physiological response. These hormones escalate heart rate and the force of contraction...
Epinephrine and Norepinephrine
The adrenal medulla releases epinephrine and norepinephrine, catecholamines that enhance and extend the sympathetic or "fight or flight" physiological response. These hormones escalate heart rate and the force of contraction...
5.8K
Drug Dosing: Geriatric Patients
212
Elderly individuals encompass a diverse population with varying degrees of age-related physiological changes. Defining the elderly presents challenges, as the geriatric population is often arbitrarily categorized as individuals older than 65. However, many individuals in this group lead active and healthy lives, with an increasing number surpassing 85 years and falling into the older elderly category. Physiological changes associated with aging impact performance capacity and homeostatic...
212
