生物医学知识的多式表示从有限的培训整个幻灯片图像和报告使用深度学习的生物医学知识
Niccolò Marini1, Stefano Marchesin2, Marek Wodzinski3
1Information Systems Institute, University of Applied Sciences Western Switzerland (HES-SO Valais), Sierre, Switzerland.
Medical image analysis
|August 18, 2024
概括
这项研究引入了一种多式联网深度学习架构,有效地将医疗图像和报告联系起来,提高数据效率并降低生物医学分析的计算成本.
科学领域:
- 生物医学信息学是生物医学信息学.
- 深度学习是一种深度学习.
- 数字病理学数字病理学
背景情况:
- 生物医学数据,包括图像和报告,提供补充信息,但有效整合具有挑战性.
- 现有的深度学习方法很难将视觉图像内容与文本报告描述联系起来.
- 这种局限性阻碍了医疗专家充分利用深度学习进行复杂分析.
研究的目的:
- 开发一个多式联网深度学习架构,以提供强大的生物医学数据表示.
- 在图像嵌入中整合细粒度文本表示,以进行增强的分析.
- 通过联合监督和自我监督的学习来解决数据稀缺问题,并创建多式模式的生物医学本体学.
主要方法:
- 开发了一个新的多式联络架构,整合了监督和自我监督的学习.
- 该架构是通过6000多个结肠全幻灯片图像 (WSI) 和相应的报告进行训练的.
- 评估涉及WSI分类,多式联运数据检索,并将文本和视觉概念联系起来.
主要成果:
- 多模式表示在结肠WSI分类中表现优于单模式方法.
- 它将数据需求减少了一半,以实现准确的性能,降低计算成本.
- 多模式数据检索和概念链接可以在没有额外的培训的情况下实现.
结论:
- 开发的架构为各种生物医学任务提供了强大的支柱,包括分类和检索.
- 它可以有效地从大型生物医学数据集中进行数据挖掘和知识提取,而无需新的专家注释.
- 创建多模式视觉本体学有可能在医学和生物医学分析方面取得重大进展.
相关概念视频
Imaging Biological Samples with Optical Microscopy
Optical microscopy uses optic principles to provide detailed images of samples. Antonie van Leeuwenhoek designed the first compound optical microscope in the 17th century to visualize blood cells, bacteria, and yeast cells. In 1830, Joseph Jackson Lister created an essentially modern light microscope. The 20th century saw the development of microscopes with enhanced magnification and resolution.
In optical microscopy, the specimen to be viewed is placed on a glass slide and clipped on the stage...
In optical microscopy, the specimen to be viewed is placed on a glass slide and clipped on the stage...
Immunofluorescence Microscopy
A fluorescence microscope uses fluorescent chromophores called fluorochromes, which can absorb energy from a light source and then emit this energy as visible light. Fluorochromes include naturally fluorescent substances (such as chlorophylls) and fluorescent stains that are added to the specimen to create contrast. Dyes such as Texas red and FITC are examples of fluorochromes. Other examples include the nucleic acid dyes 4’,6’-diamidino-2-phenylindole (DAPI), and acridine orange.
The...
The...
Confocal Fluorescence Microscopy
Confocal microscopy is an advanced microscopic technique. The prime advantage of the confocal microscope over other microscopy techniques is its ability to block the out-of-focus light from the illuminated samples using pinholes. It is widely used with fluorescence optics to obtain high-resolution, sharp contrast images. Unlike optical microscopes, confocal microscopes use a focused beam of light laser to scan the entire sample surface at different z-planes. These microscopes are, therefore,...
Super-resolution Fluorescence Microscopy
Super-resolution fluorescence microscopy (SRFM) provides a better resolution than conventional fluorescence microscopy by reducing the point spread function (PSF). PSF is the light intensity distribution from a point that causes it to appear blurred. Due to PSF, each fluorescing point appears bigger than its actual size, and it is the PSF interference of nearby fluorophores that causes the blurred image. Various approaches to achieving higher resolution through SRFM have recently been developed.
Two-Dimensional Microscopy in Microbiology
Two-dimensional (2D) microscopy encompasses a range of optical techniques that capture images within a single focal plane, offering detailed representations of microscopic structures. These techniques are essential in biological and medical research, enabling the visualization of cellular and subcellular structures with different levels of contrast and specificity.There are several major types of 2D microscopy, each with strengths and applications.Bright-Field MicroscopyBright-field microscopy...
Three-Dimensional Microscopy in Microbiology
Three-dimensional imaging techniques are essential in cell biology, allowing researchers to visualize intricate cellular structures with high resolution. Two prominent methods, Differential Interference Contrast Microscopy (DIC) and Confocal Scanning Laser Microscopy (CSLM), provide distinct advantages for imaging live and thick specimens, respectively.Differential Interference Contrast MicroscopyDIC microscopy enhances contrast in transparent, unstained samples by converting phase...


