评估人工智能语言模型的准确性和完整性,以提供有关甲氨基酸使用的信息
Belkis Nihan Coskun1, Burcu Yagiz2, Gokhan Ocakoglu3
1Division of Rheumatology, Department of Internal Medicine, Faculty of Medicine, Bursa Uludag University, Bursa, Turkey. belkisnihanseniz@hotmail.com.
Rheumatology international
|September 25, 2023
概括
与BARD和Bing相比,ChatGPT模型,特别是GPT-4,擅长回答患者关于类风湿性关节炎的甲索 (MTX) 的问题,提供更高的准确性和完整性. 然而,他们注意到了一些不准确的情况.
科学领域:
- 人工智能在医学中的应用
- 医疗信息学 医疗信息学
- 自然语言处理自然语言处理.
背景情况:
- 大型语言模型 (LLM) 越来越多地用于健康信息检索.
- 对特定疾病的LLM反应的准确性进行评估对于患者安全至关重要.
- 甲甲酸 (MTX) 是一种常见的类风湿性关节炎药物,需要明确的患者指导.
研究的目的:
- 评估ChatGPT (3.5-4),BARD和Bing在响应风湿性关节炎的甲索 (MTX) 相关查询时的准确性和完整性.
- 为了比较不同LLM在提供以患者为中心的MTX信息方面的表现.
- 识别LLM产生的医疗建议中的潜在不准确性和差距.
主要方法:
- 利用了关于MTX在类风湿性关节炎中使用的23个已确定的问题.
- 在ChatGPT (3.5-4),BARD和Bing中输入问题.
- 通过两名独立审稿人使用利克特等级来评估LLM的准确性和完整性.
主要成果:
- 聊天GPT模型实现了100%正确答案率,其中GPT-4得分100%准确,GPT-3.5得分为86.96%.
- BARD和Bing的得分较低,正确答案为73.91%,准确率为60.87%.
- 在"作用机制"和"副作用"等特定类别中,GPT-4表现出卓越的完整性 (100%),明显优于BARD和Bing.
结论:
- GPT模型,特别是GPT-4,提供了比BARD和Bing更准确和更全面的关于治疗类风湿性关节炎的甲索的信息.
- 虽然LLM显示出前景,但确定的不准确性需要谨慎和进一步验证临床使用.
- 未来的研究应该专注于提高在风湿病学患者教育的LLM可靠性.
相关概念视频
Pharmacokinetic Models: Overview
761
Pharmacokinetic models utilize mathematical analysis to achieve a detailed quantitative understanding of a drug's life cycle within the body. They are instrumental in simulating a drug's pharmacokinetic parameters, predicting drug concentrations over time, optimizing dosage regimens, linking concentrations with pharmacologic activity, and estimating potential toxicity.
There are three primary types of models: empirical, compartment, and physiological. Empirical models, with minimal...
There are three primary types of models: empirical, compartment, and physiological. Empirical models, with minimal...
761
Mutagenicity and Carcinogenicity
1.3K
Mutagenicity and carcinogenicity refer to the ability of drugs to cause genetic defects and induce cancer, respectively. The International Agency for Research on Cancer (IARC) classifies agents into four groups based on their carcinogenic potential. Group 1 agents are known human carcinogens; group 2A agents are probably carcinogenic to humans; group 3 agents lack data to support their role in carcinogenesis; and group 4 includes agents for which data support that they are not likely to be...
1.3K
Improving Translational Accuracy
2.6K
2.6K
Targets for Drug Action: Overview
6.3K
Drugs target macromolecules to modify ongoing cellular processes. Primary drug targets include receptors, ion channels, transporters, and enzymes.
Receptors are either membrane-spanning or intracellular proteins, which upon binding a ligand, get activated and transmit the signal downstream to elicit a response. Drugs bind receptors, either mimicking the action of endogenous ligands or blocking the receptor activity to bring about a modified response. Nearly 35% of approved drugs target the G...
Receptors are either membrane-spanning or intracellular proteins, which upon binding a ligand, get activated and transmit the signal downstream to elicit a response. Drugs bind receptors, either mimicking the action of endogenous ligands or blocking the receptor activity to bring about a modified response. Nearly 35% of approved drugs target the G...
6.3K
Drugs Affecting Neurotransmitter Synthesis
1.4K
Drugs affecting neurotransmitter synthesis can impact the adrenergic neuron and the synthesis of neurotransmitters. For example, α-methyltyrosine and carbidopa target specific enzymes involved in catecholamine synthesis. α-methyltyrosine inhibits the enzyme tyrosine hydroxylase, which converts tyrosine into dopamine. By blocking this enzyme, α-methyltyrosine reduces dopamine production and other catecholamines. Carbidopa, on the other hand, inhibits the enzyme dopa decarboxylase,...
1.4K
Drug Biotransformation: Overview
2.4K
Pharmaceutical substances known as xenobiotics are predominantly lipophilic and nonionized. This enables them to permeate lipid bilayers, such as cell membranes, and interact with intracellular target receptors. Lipophilic drugs have an advantage in crossing biological barriers and reaching their intended sites of action. However, lipophilic drugs often have a restricted capacity for renal expulsion or elimination from the body. When these drugs enter the kidneys and undergo glomerular...
2.4K


