Related Experiment Video
Updated: May 8, 2026
Application of Deep Learning-Based Medical Image Segmentation via Orbital Computed Tomography
Published on: November 30, 2022
Deep Learning Models Connecting Images and Text: A Primer for Radiologists.
An Ni Wu1, Merve Kulbay1, Phillip M Cheng1
1From the Departments of Radiology, Radiation Oncology, and Nuclear Medicine, Centre hospitalier de l'Université de Montréal, Université de Montréal, 1000 rue Saint-Denis, D03.5431, Montreal, QC, Canada H2X 0C1 (A.N.W., A.C.C., L.L.G., A.T.); Centre de recherche du Centre hospitalier de l'Université de Montréal, Montreal, Quebec, Canada (A.N.W., M.K., L.L.G., E.M., I.B.A., A.T.); Department of Ophthalmology and Visual Sciences, McGill University, Montreal, Quebec, Canada (M.K.); Department of Radiology, Keck School of Medicine of the University of Southern California, Los Angeles, Calif (P.M.C.); Department of Medical Imaging, CISSS Lanaudiére, Université Laval, Joliette, Quebec, Canada (A.C.C.); AFX Medical, Montreal, Quebec, Canada (G.C.); Department of Medical Imaging, Western University, London, Ontario, Canada (J.C.); École de Technologie Supérieure, Montreal, Quebec, Canada (I.B.A.); and Institute of Biomedical Engineering, Université de Montréal, Montreal, Quebec, Canada (A.T.).
Deep learning models are advancing the connection between medical images and text, streamlining radiology workflows. These innovations promise improved diagnostic accuracy and efficiency in clinical practice.
Area of Science:
- Artificial Intelligence in Medicine
- Medical Imaging Informatics
Background:
- Radiology practice relies on radiologists interpreting medical images and generating text reports.
- Recent advancements in deep learning facilitate the integration of image and text data.
Purpose of the Study:
- To explore the potential of deep learning models in connecting medical images and text.
- To categorize models that link image and text data.
- To identify clinical applications and benefits for radiology workflows.
Main Methods:
- Review of recent technical developments in data embedding, self-supervised learning, zero-shot learning, and transformer architectures.
- Categorization of image-text models into text-image alignment, image-to-text, text-to-image, and multimodal approaches.
Main Results:
- Deep learning models can align text and images, generate descriptions from images, create images from text, and integrate multimodal data.
- Potential applications include automated image captioning, preliminary report generation, and educational image creation.
Conclusions:
- These AI-driven advancements can enhance radiology by prioritizing cases, streamlining workflows, and improving diagnostic accuracy.
- The integration of deep learning in radiology holds significant promise for clinical practice.
