ConTEXTual Net:用于肺胸部细分的多模式视觉语言模型
Zachary Huemann1, Xin Tie2, Junjie Hu3
1Department of Radiology, University of Wisconsin-Madison, Madison, WI, 53705, USA. zhuemann@wisc.edu.
Journal of imaging informatics in medicine
|March 15, 2024
概括
一个新的模型,ContexTual Net,使用放射学报告文本来改善胸部X射线中的肺胸部细分. 这种视觉语言方法与人类专家的准确性相匹配,优于其他人工智能模型.
科学领域:
- 医疗成像医学成像
- 人工智能的人工智能
- 自然语言处理自然语言处理.
背景情况:
- 放射学报告包含有关疾病特征的描述性文本.
- 多模式学习在整合文本和图像数据方面表现有前途.
- 精确的肺胸部细分对于患者的诊断和治疗至关重要.
研究的目的:
- 开发一种新的视觉语言模型,ContexTual Net,以改善肺胸部细分.
- 从放射学报告中利用描述性文本指导医学图像分析.
- 评估ContexTual Net的性能与现有模型和人类变异性相比.
主要方法:
- 提出了ContexTual Net,这是一个视觉语言模型,将自由形式的放射学报告与胸部X射线图相结合.
- 使用预先训练的语言模型从报告中提取特征.
- 在语言特征和卷积神经网络的中间嵌入之间实现了交叉注意.
- 在CANDID-PTX数据集中接受培训和验证,其中包括3196例阳性肺胸病例.
主要成果:
- ConTEXTual Net获得了0.716±0.016的子得分,相当于读者之间的变化 (0.712±0.044).
- 超越仅视觉模型 (例如,Swin UNETR,nnUNet) 和竞争视觉语言模型 (LAVT).
- 废弃研究证实了文本信息是性能增长的主要驱动因素.
结论:
- ConTEXTual Net有效地利用放射学报告文本来提高肺胸部细分的准确性.
- 该模型的性能接近人类专家协议水平.
- 图像-文本一致性至关重要,某些增强可以对性能产生负面影响.
相关概念视频
01:26Pneumothorax-I
202
A pneumothorax is a condition where air builds up in the space between the lung and the chest wall, causing the lung to collapse. This condition arises when air enters the space between the parietal and visceral pleura, disrupting the negative pressure essential for lung inflation. This can lead to a partial or complete collapse of the lung.
Pneumothorax can be even further classified as spontaneous, traumatic, and tension pneumothorax.
Pneumothorax can be even further classified as spontaneous, traumatic, and tension pneumothorax.
202
01:27Pneumothorax-II
145
Pneumothorax is a medical condition defined by the buildup of air in the pleural space between the lungs and the chest wall. This accumulation of air can lead to partial or complete lung collapse, resulting in a range of clinical manifestations. Understanding the clinical presentation and effective management strategies is crucial for healthcare professionals in providing timely and appropriate care to individuals with pneumothorax.
Clinical Manifestations:
Clinical Manifestations:
145
01:29Assessment of Ventilation II: Respiratory Depth and Rhythm
1.5K
Respiratory Depth
Respiratory depth measures the volume of air inhaled or exhaled during a breath. It can vary from shallow to deep and typically remains consistent when a person is at rest or asleep. Occasionally, individuals will automatically inhale deeply, known as sighing, which inflates the lungs with more air than normal breathing.
To assess respiratory depth, observe the degree of chest excursion or movement:
Respiratory depth measures the volume of air inhaled or exhaled during a breath. It can vary from shallow to deep and typically remains consistent when a person is at rest or asleep. Occasionally, individuals will automatically inhale deeply, known as sighing, which inflates the lungs with more air than normal breathing.
To assess respiratory depth, observe the degree of chest excursion or movement:
1.5K