为可靠的癌症信息提供生成性AI聊天机器人:评估新兴大型语言模型的网络搜索,多语言和参考能力
Bradley D Menz1, Natansh D Modi1, Ahmad Y Abuhelwa2
1College of Medicine and Public Health, Flinders Health and Medical Research Institute, Flinders University, Adelaide, Australia.
概括
大型语言模型 (LLM) 对健康信息有希望,但需要进一步评估. 持续的研究是必不可少的,以确保准确的,安全的,可访问的多语言的健康内容从LLMs.
科学领域:
- 人工智能的人工智能
- 医疗信息学 医疗信息学
- 医学沟通 医学沟通
背景情况:
- 大型语言模型 (LLM) 提供了高级功能,如实时网络搜索和多语言支持.
- 确保LLM提供的健康信息的安全性和准确性是一个关键的挑战.
- 之前的评估强调了对LLM产生的健康内容进行严格评估的需要.
研究的目的:
- 评估七个公开可访问的LLM提供的健康信息的安全性和质量.
- 评估跨多种语言的LLM绩效,并确定潜在的差异.
- 检查LLM产生的健康反应的引用实践和可读性.
主要方法:
- 七个LLM (ChatGPT,Co-Pilot,Gemini,MetaAI,Claude,Grok,Perplexity) 进行了测试. 这些LLM包括:
- 在八种语言 (英语,法语,中文,泰语,印度语,尼泊尔语,越南语,阿拉伯语) 中使用了三个简单的与癌症有关的查询.
- 对临床幻觉,参考有效性,参考质量 (例如.com链接) 和可读性进行了分析.
主要成果:
- 在42个英语答案中,没有发现临床意义上的幻觉.
- 在294个非英语答案中,有7个含有临床意义上的幻觉.
- 48%的所有回复包括有效的引用,其中39%的英语引用是.com链接.
- 许多英语和非英语的答案超过了八年级的阅读水平.
结论:
- 在提供健康信息方面,LLM在提供健康信息方面取得了重大进展,但存在持续的差距.
- 多语言准确性,可靠的引用和适当的可读性仍然是需要改进的领域.
- 持续的基准测试至关重要,以确保LLMs安全地满足全球健康信息标准.
相关概念视频
Non-equilibrium in the Cell
4.1K
An important concept in studying metabolism and energy is that of chemical equilibrium. Most chemical reactions are reversible. They can proceed in both directions, releasing energy into their environment in one direction, and absorbing it from the environment in the other direction. The same is true for the chemical reactions involved in cell metabolism, such as the breaking down and building up of proteins into and from individual amino acids, respectively. Reactants within a closed system...
4.1K
Mouse Models of Cancer Study
5.5K
Mice have long served as models for studying human biology and pathology because of their phylogenetic and physiological similarity with humans. They are also easy to maintain and breed in the laboratory, and hence, many inbred strains are now available for research. Studies on mice have contributed immeasurably to our understanding of cancer biology.
The development of transgenic, knockout, and knock-in mice has led to an exponential increase in their use as model organisms in research,...
The development of transgenic, knockout, and knock-in mice has led to an exponential increase in their use as model organisms in research,...
5.5K
Combination Therapies and Personalized Medicine
4.8K
Combining two or more treatment methods increases the life span of cancer patients while reducing damage to vital organs or tissue from the overuse of a single treatment. Combination therapy also targets different cancer-inducing pathways, thus reducing the chances of developing resistance to treatment.
The combination of the drug acetazolamide and sulforaphane is a good example of combination therapy to treat cancer. The cells in the interior of a large tumor often die due to the hypoxic and...
The combination of the drug acetazolamide and sulforaphane is a good example of combination therapy to treat cancer. The cells in the interior of a large tumor often die due to the hypoxic and...
4.8K
Cancer Survival Analysis
321
Cancer survival analysis focuses on quantifying and interpreting the time from a key starting point, such as diagnosis or the initiation of treatment, to a specific endpoint, such as remission or death. This analysis provides critical insights into treatment effectiveness and factors that influence patient outcomes, helping to shape clinical decisions and guide prognostic evaluations. A cornerstone of oncology research, survival analysis tackles the challenges of skewed, non-normally...
321
What is Cancer?
10.5K
Cells and tissues must meticulously coordinate their activities for the normal functioning of the human body. Therefore, they exhibit socially responsible behavior - resting, growing, dividing, differentiating, or dying - for the organism’s benefit. Cancer arises when cells divide uncontrollably and invade other tissues or organs.
Although people have known about cancer for centuries, it was only in 1761 that Giovanni Morgagni of Padua performed a detailed autopsy of...
Although people have known about cancer for centuries, it was only in 1761 that Giovanni Morgagni of Padua performed a detailed autopsy of...
10.5K
Cancer Therapies
7.5K
Cancer therapies are various modes of treatment, such as surgery, radiation therapy, and chemotherapy that are administered to cancer patients.
However, cancer treatments can pose several challenges, as therapies used to kill cancer cells are generally also toxic to normal cells. Moreover, cancer cells mutate rapidly and can develop resistance to chemical agents or radiation therapy. Besides, all types of cancer cells may not respond to the same therapy. Some cancer cells respond to one...
However, cancer treatments can pose several challenges, as therapies used to kill cancer cells are generally also toxic to normal cells. Moreover, cancer cells mutate rapidly and can develop resistance to chemical agents or radiation therapy. Besides, all types of cancer cells may not respond to the same therapy. Some cancer cells respond to one...
7.5K


