XRaySwinGen:使用多式模式模型进行X射线检查的自动医疗报告
Gilvan Veras Magalhães1, Roney L de S Santos1, Luis H S Vogado1
1Departamento de Computação, Universidade Federal do Piauí, Teresina, Brazil.
Heliyon
|April 1, 2024
概括
这项研究引入了一个人工智能模型,用于从X射线生成详细的放射学报告,提高诊断准确度. 多式联络方法增强了医疗图像标题,以获得更好的患者状况洞察力.
科学领域:
- 医学成像和人工智能 医学成像和人工智能
- 计算机视觉和自然语言处理
背景情况:
- 手动生成放射学报告是耗时且容易出现错误的.
- 医疗成像中的当前人工智能往往专注于分类,错过了关键的患者状况细节.
- 从X射线生成描述性报告仍然是人工智能面临的挑战.
研究的目的:
- 开发一种多式人工智能模型,从X射线图像中生成详细,描述性的放射学报告.
- 通过专注于全面的报告生成,克服当前AI医疗图像标题的局限性.
- 通过先进的AI技术,提高医疗报告的准确性和效率.
主要方法:
- 使用了多式模式模型,将计算机视觉 (Swin Transformer) 用于图像编码和自然语言处理 (GPT-2) 用于文本解码.
- 实现了Swin变压器用于层次图像表示,增强感知而不增加计算成本.
- 员工交叉关注层和双语培训 (葡萄牙语PT-BR和英语) 提供了强大的报告生成.
主要成果:
- 在拟议的数据库上实现了有前途的性能,ROUGE-L为0.748和METEOR为0.741.
- 在NIH胸部X射线数据集上证明了有效性,产生ROUGE-L的0.404和METEOR的0.393.
- 该模型成功地从X射线图像中生成了描述性的文本报告.
结论:
- 拟议的多式联动人工智能模型有效生成详细的放射学报告,解决当前医疗图像标题的局限性.
- 斯温变压器和GPT-2的创新性使用为人工智能驱动的医疗报告生成提供了高效和有效的方法.
- 双语培训和有前途的结果表明该模型在改善放射学工作流程和诊断能力方面的潜力.
相关概念视频
X-ray Imaging
5.5K
German physicist Wilhelm Röntgen (1845–1923) was experimenting with electrical current when he discovered that a mysterious and invisible "ray" would pass through his flesh but leave an outline of his bones on a screen coated with a metal compound. In 1895, Röntgen made the first durable record of the internal parts of a living human: an "X-ray" image (as it came to be called) of his wife’s hand. Scientists worldwide quickly began their own experiments with...
5.5K
Computed Tomography
4.5K
Tomography refers to imaging by sections. Computed tomography (CT) is a non-invasive imaging technique that uses computers to analyze several cross-sectional X-rays to reveal minute details about structures in the body.
The technique was invented in the 1970s and is based on the principle that as X-rays pass through the body, they are absorbed or reflected at different levels. In the technique, a patient lies on a motorized platform while a computerized axial tomography (CAT) scanner rotates...
The technique was invented in the 1970s and is based on the principle that as X-rays pass through the body, they are absorbed or reflected at different levels. In the technique, a patient lies on a motorized platform while a computerized axial tomography (CAT) scanner rotates...
4.5K
Imaging Studies for Cardiovascular System III: X-Ray
180
The most common cardiovascular diagnostic test is an X-ray. It produces images of the heart, blood vessels, and adjacent structures.
Definition and Purpose
An X-ray, or radiograph, is a non-invasive method that uses ionizing radiation to take images of internal structures. It is mainly used in cardiac imaging to examine the heart, lungs, and major blood vessels, aiming to identify abnormalities in the heart's size, shape, and position, such as heart failure, congenital defects, and vascular...
Definition and Purpose
An X-ray, or radiograph, is a non-invasive method that uses ionizing radiation to take images of internal structures. It is mainly used in cardiac imaging to examine the heart, lungs, and major blood vessels, aiming to identify abnormalities in the heart's size, shape, and position, such as heart failure, congenital defects, and vascular...
180
Positron Emission Tomography
4.2K
Positron emission tomography (PET) is a medical imaging technique involving radiopharmaceuticals — substances that emit short-lived radiation. Although the first PET scanner was introduced in 1961, it took 15 more years before radiopharmaceuticals were combined with the technique and revolutionized its potential.
One of the main requirements of a PET scan is a positron-emitting radioisotope, which is produced in a cyclotron and then attached to a substance used by the part of the body...
One of the main requirements of a PET scan is a positron-emitting radioisotope, which is produced in a cyclotron and then attached to a substance used by the part of the body...
4.2K
Magnetic Resonance Imaging
5.1K
Magnetic resonance imaging (MRI) is a noninvasive medical imaging technique based on a phenomenon of nuclear physics discovered in the 1930s, in which matter exposed to magnetic fields and radio waves was found to emit radio signals. In 1970, a physician and researcher named Raymond Damadian noticed that malignant (cancerous) tissue gave off different signals than normal body tissue. He applied for a patent for the first MRI scanning device in clinical use by the early 1980s. The early MRI...
5.1K


