一个多式视觉语言基础模型用于计算眼科
Danli Shi1,2, Weiyi Zhang3, Jiancheng Yang4
1School of Optometry, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR, China. danli.shi@polyu.edu.hk.
NPJ digital medicine
|June 20, 2025
概括
EyeCLIP是一款新的多式人工智能模型,使用277万张图像改进了早期眼病检测. 它擅长识别罕见疾病和各种眼科疾病.
科学领域:
- 眼科医生 眼科 眼科
- 人工智能的人工智能
- 医疗成像医学成像
背景情况:
- 早期发现眼病对于预防视力丧失至关重要.
- 当前的人工智能模型经常使用单一的数据类型,并与罕见的条件作斗争.
- 人工智能需要整合多视图信息,以便全面诊断眼病.
研究的目的:
- 介绍EyeCLIP,眼科的多式视觉语言基础模型.
- 解决单模人工智能在检测罕见和常见的眼病方面的局限性.
- 为了提高诊断能力,利用大量的数据集和新的预训练.
主要方法:
- 经过 EyeCLIP 的训练,EyeCLIP 在 11 种模式的眼科图像中使用了 277 万张眼科图像,并提供了临床文本.
- 采用了一种预训练策略,结合了自我监督的重建,多式模式对比学习和图像-文本对比学习.
- 对各种眼科人工智能任务的14个基准数据集进行性能评估.
主要成果:
- EyeCLIP在疾病分类,视觉问题答案和交叉模式检索方面表现强.
- 获得了强大的少数射击和零射击学习能力,这对于长尾疾病分布至关重要.
- 从医学图像中检测眼睛和全身疾病的有效性.
结论:
- EyeCLIP代表了眼科多式人工智能的重大进步.
- 该模型处理各种数据和罕见疾病的能力为现实世界的临床应用提供了潜力.
- EyeCLIP可以弥合当前诊断工具的差距,通过早期检测改善患者的治疗结果.
相关概念视频
01:26Visual System
706
Light enters the eye through the cornea, a transparent, dome-shaped surface covering the surface of the eyeball that helps to direct and focus incoming light. This light is then channeled toward the pupil, an adjustable opening whose size is controlled by the iris. The iris, a pigmented muscle, regulates the amount of light entering the eye by contracting or dilating the pupil, thereby ensuring optimal light levels for clear vision.
Once through the pupil, the light passes through the lens, a...
Once through the pupil, the light passes through the lens, a...
706
01:25Glaucoma: Overview
789
Glaucoma is an eye condition characterized by increased intraocular pressure that damages the retina and optic nerve, leading to irreversible blindness if left untreated. The human eye has various components, including the cornea, iris, pupil, lens, and optic nerve. Aqueous humor is secreted by the epithelium of the ciliary body in the posterior chamber and flows through the trabecular meshwork and canal of Schlemm, maintaining normal intraocular pressure. The trabecular meshwork and the canal...
789
01:24Vision
55.4K
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
55.4K
01:18Imaging Biological Samples with Optical Microscopy
5.5K
Optical microscopy uses optic principles to provide detailed images of samples. Antonie van Leeuwenhoek designed the first compound optical microscope in the 17th century to visualize blood cells, bacteria, and yeast cells. In 1830, Joseph Jackson Lister created an essentially modern light microscope. The 20th century saw the development of microscopes with enhanced magnification and resolution.
In optical microscopy, the specimen to be viewed is placed on a glass slide and clipped on the stage...
In optical microscopy, the specimen to be viewed is placed on a glass slide and clipped on the stage...
5.5K