紧的视觉语言模型可以通过层特定的多式联络学习实现高效和可解释的光学连贯断层扫描
Tania Haghighi1, Sina Gholami1, Jared Todd Sokol2
1Department of Electrical and Computer Engineering, University of North Carolina at Charlotte, Charlotte, NC, USA.
Communications medicine
|December 27, 2025
概括
新的人工智能模型LO-VLM从视网膜OCT扫描生成准确的临床叙述. 它在文本生成和疾病分类方面都胜过现有的模型,提高了海外国家和地区的解释效率.
科学领域:
- 眼科医生 眼科 眼科
- 人工智能的人工智能
- 医疗成像医学成像
背景情况:
- 来自光学连贯断层扫描 (OCT) B扫描的准确临床叙述对于诊断视网膜疾病至关重要.
- 目前的算法很难将视觉特征与领域专业知识相结合,以实现有效的解释.
研究的目的:
- 开发一个高效的人工智能模型,从OCT扫描生成临床叙述.
- 为了提高视网膜成像中疾病分类的准确性.
主要方法:
- 一个多式数据集由40,000个OCTB扫描进行了策划,并与专家验证的摘要配对.
- 介绍了LO-VLM,这是一个紧的视觉语言模型 (VLM),用于概要生成和分类的解剖指导.
- 基准LO-VLM与RetinaVLM,LLaVA-Med以及一个ViT模型进行比较.
主要成果:
- 在盲目专家评估中,LO-VLM叙述获得了8.5/10的平均得分,明显优于RetinaVLM (5.5/10).
- 实现了80.3%的SBERT相似性和71.5%的BERTScore F1,超过了专业VLM基线.
- 在疾病分类中达到96%的准确性,超过ViT的13%和医疗VLM的62%以上.
结论:
- LO-VLM为OCT解释中高效和可解释的AI模型提供了一个范例.
- 该模型成功地将计算效率与临床叙事生成和疾病分类中的高性能相结合.
相关概念视频
Imaging Biological Samples with Optical Microscopy
8.7K
Optical microscopy uses optic principles to provide detailed images of samples. Antonie van Leeuwenhoek designed the first compound optical microscope in the 17th century to visualize blood cells, bacteria, and yeast cells. In 1830, Joseph Jackson Lister created an essentially modern light microscope. The 20th century saw the development of microscopes with enhanced magnification and resolution.
In optical microscopy, the specimen to be viewed is placed on a glass slide and clipped on the stage...
In optical microscopy, the specimen to be viewed is placed on a glass slide and clipped on the stage...
8.7K
Confocal Fluorescence Microscopy
19.9K
Confocal microscopy is an advanced microscopic technique. The prime advantage of the confocal microscope over other microscopy techniques is its ability to block the out-of-focus light from the illuminated samples using pinholes. It is widely used with fluorescence optics to obtain high-resolution, sharp contrast images. Unlike optical microscopes, confocal microscopes use a focused beam of light laser to scan the entire sample surface at different z-planes. These microscopes are, therefore,...
19.9K
Computed Tomography
7.9K
Tomography refers to imaging by sections. Computed tomography (CT) is a non-invasive imaging technique that uses computers to analyze several cross-sectional X-rays to reveal minute details about structures in the body.
The technique was invented in the 1970s and is based on the principle that as X-rays pass through the body, they are absorbed or reflected at different levels. In the technique, a patient lies on a motorized platform while a computerized axial tomography (CAT) scanner rotates...
The technique was invented in the 1970s and is based on the principle that as X-rays pass through the body, they are absorbed or reflected at different levels. In the technique, a patient lies on a motorized platform while a computerized axial tomography (CAT) scanner rotates...
7.9K
Imaging Studies III: Computed Tomography
251
DefinitionComputed Tomography (CT) of the genitourinary (GU) tract is a non-invasive imaging modality that utilizes X-rays and computer processing to generate detailed cross-sectional images of the urinary system, encompassing the kidneys, ureters, bladder, and adjacent structures such as the adrenal glands.PurposeCT scans of the GU tract serve several diagnostic and therapeutic purposes, including:Diagnosis of Urinary Tract Diseases: Detects kidney stones, tumors, cysts, and congenital...
251
Phase Contrast and Differential Interference Contrast Microscopy
11.9K
Phase-Contrast Microscopes
In-phase-contrast microscopes, interference between light directly passing through a cell and light refracted by cellular components is used to create high-contrast, high-resolution images without staining. It is the oldest and simplest type of microscope that creates an image by altering the wavelengths of light rays passing through the specimen. Altered wavelength paths are created using an annular stop in the condenser. The annular stop produces a hollow cone of...
In-phase-contrast microscopes, interference between light directly passing through a cell and light refracted by cellular components is used to create high-contrast, high-resolution images without staining. It is the oldest and simplest type of microscope that creates an image by altering the wavelengths of light rays passing through the specimen. Altered wavelength paths are created using an annular stop in the condenser. The annular stop produces a hollow cone of...
11.9K
Depth Perception and Spatial Vision
1.7K
Depth perception is the ability to perceive objects three-dimensionally. It relies on two types of cues: binocular and monocular. Binocular cues depend on the combination of images from both eyes and how the eyes work together. Since the eyes are in slightly different positions, each eye captures a slightly different image. This disparity between images, known as binocular disparity, helps the brain interpret depth. When the brain compares these images, it determines the distance to an object.
1.7K


