大型语言模型在回答免疫瘤学问题的比较:一个横截面研究
Giovanni Maria Iannantuono1, Dara Bracken-Clarke2, Fatima Karzai1
1Genitourinary Malignancies Branch, Center for Cancer Research, National Cancer Institute, National Institutes of Health, Bethesda, MD, United States.
medRxiv : the preprint server for health sciences
|December 11, 2023
概括
聊天GPT-4和聊天GPT-3.5显示作为免疫瘤学工具的承诺,超过了谷歌Bard. 然而,由于潜在的不准确性,所有大型语言模型都需要专家验证.
科学领域:
- 在瘤学中使用人工智能
- 在医疗保健中的自然语言处理.
背景情况:
- 大型语言模型 (LLM) 为癌症患者和医疗保健提供者提供了教育和管理工具的潜力.
- 调查LLM在理解和生成人类可读的文本瘤应用程序的能力.
研究的目的:
- 评估ChatGPT-4,ChatGPT-3.5和谷歌巴德在回答关键免疫瘤学领域的问题方面的表现.
- 为了比较不同LLM的答案的准确性,相关性和可重复性.
主要方法:
- 一项横截面研究,涉及60个关于免疫瘤学的开放式问题 (机制,指示,毒性,预后).
- 手动向ChatGPT-4,ChatGPT-3.5和谷歌巴德提交问题.
- 两个专家对LLM答案进行了独立的审查和评估.
主要成果:
- 聊天GPT-4和聊天GPT-3.5回答了所有问题,而谷歌巴德回答了53.3% (p <0.0001).
- 与谷歌Bard (43.8%) 相比,ChatGPT-4和ChatGPT-3.5的准确性 (75.4%,58.5%) 和相关性 (71.9%,77.4%) 更高.
- 对于ChatGPT模型来说,响应可重复性明显高 (95%,88.3%),而不是谷歌巴德 (50%) (p <0.0001).
结论:
- 在免疫瘤学领域,ChatGPT-4和ChatGPT-3.5显示出作为有价值的工具的巨大潜力.
- 谷歌巴德在回答免疫瘤学问题方面表现相对较差.
- 所有评估的LLM都需要专家验证,以减轻不准确或不完整信息的风险.
相关概念视频
Tumor Immunotherapy
529
Immunotherapy is a treatment that boosts or manipulates the immune system to fight diseases, including cancer. For instance, by stimulating an immune response through vaccinations against viruses that cause cancers, like hepatitis B virus and human papillomavirus, these diseases can be prevented. Nonetheless, some cancer cells can avoid the immune system due to their rapid mutation and division. The immune response to many cancers involves three phases: elimination, equilibrium, and escape.
529
Mouse Models of Cancer Study
5.6K
Mice have long served as models for studying human biology and pathology because of their phylogenetic and physiological similarity with humans. They are also easy to maintain and breed in the laboratory, and hence, many inbred strains are now available for research. Studies on mice have contributed immeasurably to our understanding of cancer biology.
The development of transgenic, knockout, and knock-in mice has led to an exponential increase in their use as model organisms in research,...
The development of transgenic, knockout, and knock-in mice has led to an exponential increase in their use as model organisms in research,...
5.6K
Cytotoxic T Cells-mediated Immune Response
923
Cytotoxic T cells are a vital component of the immune system. They have the remarkable ability to identify and target antigens on infected or abnormal cells. These antigens often originate from intracellular pathogens such as viruses or abnormal proteins cancer cells produce.
Immunological surveillance is the ability of immune cells to monitor and eliminate infected cells with intracellular pathogens, neoplastically transformed cells, and cells with non-self antigens. Cytotoxic T cells and NK...
Immunological surveillance is the ability of immune cells to monitor and eliminate infected cells with intracellular pathogens, neoplastically transformed cells, and cells with non-self antigens. Cytotoxic T cells and NK...
923
Combination Therapies and Personalized Medicine
4.9K
Combining two or more treatment methods increases the life span of cancer patients while reducing damage to vital organs or tissue from the overuse of a single treatment. Combination therapy also targets different cancer-inducing pathways, thus reducing the chances of developing resistance to treatment.
The combination of the drug acetazolamide and sulforaphane is a good example of combination therapy to treat cancer. The cells in the interior of a large tumor often die due to the hypoxic and...
The combination of the drug acetazolamide and sulforaphane is a good example of combination therapy to treat cancer. The cells in the interior of a large tumor often die due to the hypoxic and...
4.9K


