在上下文学习使多式大型语言模型能够对癌症病理图像进行分类
Dyke Ferber1,2,3, Georg Wölflein4, Isabella C Wiest3,5
1National Center for Tumor Diseases (NCT), Heidelberg University Hospital, Heidelberg, Germany.
Nature communications
|November 21, 2024
概括
像GPT-4V这样的大型视觉语言模型可以使用上下文学习进行医学图像分类,以最小的数据匹配或超过专业模型. 这种方法使医疗专家的AI民主化,特别是在数据稀缺的地方.
科学领域:
- 人工智能的人工智能
- 医疗成像医学成像
- 计算病理学计算病理学
背景情况:
- 医学图像分类通常需要广泛的,特定任务的数据集来训练深度学习模型,这个过程在计算上密集,在技术上具有挑战性.
- 在语境学习,一种模型在没有参数更新的情况下从提示中学习的方法,已在自然语言处理中建立,但在医学图像分析中未得到充分探索.
- 在专业医疗领域,注释数据的稀缺性对开发有效的AI诊断工具构成重大障碍.
研究的目的:
- 系统地评估使用上下文学习进行癌症图像处理的具有视觉功能的生成预训练变压器4 (GPT-4V) 的有效性.
- 评估GPT-4V在三个关键的组织病理学任务上的表现:结直肠癌组织亚型,结肠多亚型和淋巴结部分乳腺瘤检测.
- 确定大视觉语言模型的上下文学习是否可以作为医学图像分析中传统深度学习方法的可行替代方案.
主要方法:
- 评估GPT-4V在三个不同的癌症组织病理学数据集上的上下文学习能力.
- 将GPT-4V的性能与针对相同任务训练的专用神经网络进行比较.
- 评估医疗图像分类中有效的上下文学习所需的样本数量.
主要成果:
- 使用GPT-4V的上下文学习在所有评估的组织病理学任务中表现出与专业深度学习模型相比或超过的性能.
- GPT-4V以最小的样本数量取得了这些结果,突出显示了上下文学习的效率.
- 该研究证实,在非域特定数据上训练的通用AI模型可以有效地应用于医疗图像处理任务.
结论:
- 大视觉语言模型,如GPT-4V,可以成功地应用于医疗图像处理任务,特别是在组织病理学中,使用上下文学习.
- 语境学习为医学图像分析的传统模型培训提供了一个强大的,数据效率高的替代方案,使人工智能可访问性民主化.
- 这种方法对医学专家来说具有重大前景,特别是在资源有限的环境或缺乏注释数据的地区,这使得人工智能工具的广泛采用成为可能.
相关概念视频
02:43Mouse Models of Cancer Study
5.5K
Mice have long served as models for studying human biology and pathology because of their phylogenetic and physiological similarity with humans. They are also easy to maintain and breed in the laboratory, and hence, many inbred strains are now available for research. Studies on mice have contributed immeasurably to our understanding of cancer biology.
The development of transgenic, knockout, and knock-in mice has led to an exponential increase in their use as model organisms in research,...
The development of transgenic, knockout, and knock-in mice has led to an exponential increase in their use as model organisms in research,...
5.5K
02:17The Tumor Microenvironment
6.5K
Every normal cell or tissue is embedded in a complex local environment called stroma, consisting of different cell types, a basal membrane, and blood vessels. As normal cells mutate and develop into cancer cells, their local environment also changes to allow cancer progression. The tumor microenvironment (TME) consists of a complex cellular matrix of stromal cells and the developing tumor. The cross-talk between cancer cells and surrounding stromal cells is critical to disrupt normal tissue...
6.5K
01:27Tumor Immunotherapy
480
Immunotherapy is a treatment that boosts or manipulates the immune system to fight diseases, including cancer. For instance, by stimulating an immune response through vaccinations against viruses that cause cancers, like hepatitis B virus and human papillomavirus, these diseases can be prevented. Nonetheless, some cancer cells can avoid the immune system due to their rapid mutation and division. The immune response to many cancers involves three phases: elimination, equilibrium, and escape.
480
02:18Cancer
48.0K
Cancers arise due to mutations in genes involved in the regulation of cell division, which leads to unrestricted cell proliferation. Modern science and medicine have made great strides in the understanding and treatment of cancer, including eradicating cancer in some patients. However, there is still no cure for cancer. This is largely due to the fact that cancer is a large group of many diseases.
48.0K
02:12What is Cancer?
10.5K
Cells and tissues must meticulously coordinate their activities for the normal functioning of the human body. Therefore, they exhibit socially responsible behavior - resting, growing, dividing, differentiating, or dying - for the organism’s benefit. Cancer arises when cells divide uncontrollably and invade other tissues or organs.
Although people have known about cancer for centuries, it was only in 1761 that Giovanni Morgagni of Padua performed a detailed autopsy of...
Although people have known about cancer for centuries, it was only in 1761 that Giovanni Morgagni of Padua performed a detailed autopsy of...
10.5K
02:49Cancer Therapies
7.6K
Cancer therapies are various modes of treatment, such as surgery, radiation therapy, and chemotherapy that are administered to cancer patients.
However, cancer treatments can pose several challenges, as therapies used to kill cancer cells are generally also toxic to normal cells. Moreover, cancer cells mutate rapidly and can develop resistance to chemical agents or radiation therapy. Besides, all types of cancer cells may not respond to the same therapy. Some cancer cells respond to one...
However, cancer treatments can pose several challenges, as therapies used to kill cancer cells are generally also toxic to normal cells. Moreover, cancer cells mutate rapidly and can develop resistance to chemical agents or radiation therapy. Besides, all types of cancer cells may not respond to the same therapy. Some cancer cells respond to one...
7.6K