Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Predicting Knee osteoarthritis progression using explainable machine learning and clinical imaging data.

Osteoarthritis and cartilage open·2026
Same author

Beyond the LUMIR challenge: The pathway to foundational registration models.

Medical image analysis·2026
Same author

Can Crowdsourced Annotations Improve AI-based Congestion Scoring For Bedside Lung Ultrasound?

Medical image computing and computer-assisted intervention : MICCAI ... International Conference on Medical Image Computing and Computer-Assisted Intervention·2026
Same author

Stretcher: a learning-based framework for deformation-robust keypoint descriptors.

International journal of computer assisted radiology and surgery·2026
Same author

A 3-D Cross-Modal Keypoint Descriptor for MR-US Matching and Registration.

IEEE transactions on medical imaging·2026
Same author

Intraoperative Registration by Cross-Modal Inverse Neural Rendering.

Medical image computing and computer-assisted intervention : MICCAI ... International Conference on Medical Image Computing and Computer-Assisted Intervention·2025

Related Experiment Video

Updated: Jan 15, 2026

Author Spotlight: An Efficient and Robust Software for Automated Fusion of Multiple Preclinical Imaging Modalities
07:13

Author Spotlight: An Efficient and Robust Software for Automated Fusion of Multiple Preclinical Imaging Modalities

Published on: October 27, 2023

1.6K

Unified Cross-Modal Medical Image Synthesis With Hierarchical Mixture of Product-of-Experts.

Reuben Dorent, Nazim Haouchine, Alexandra Golby

    IEEE Transactions on Pattern Analysis and Machine Intelligence
    |October 8, 2025
    PubMed
    Summary

    This study introduces a novel deep learning model, the multimodal hierarchical variational auto-encoder (MMHVAE), for synthesizing missing medical images. The MMHVAE effectively reconstructs high-resolution images from incomplete multimodal data, improving cross-modal image synthesis.

    More Related Videos

    A Pipeline for 3D Multimodality Image Integration and Computer-assisted Planning in Epilepsy Surgery
    09:41

    A Pipeline for 3D Multimodality Image Integration and Computer-assisted Planning in Epilepsy Surgery

    Published on: May 20, 2016

    12.7K
    Application of Deep Learning-Based Medical Image Segmentation via Orbital Computed Tomography
    04:48

    Application of Deep Learning-Based Medical Image Segmentation via Orbital Computed Tomography

    Published on: November 30, 2022

    3.3K

    Related Experiment Videos

    Last Updated: Jan 15, 2026

    Author Spotlight: An Efficient and Robust Software for Automated Fusion of Multiple Preclinical Imaging Modalities
    07:13

    Author Spotlight: An Efficient and Robust Software for Automated Fusion of Multiple Preclinical Imaging Modalities

    Published on: October 27, 2023

    1.6K
    A Pipeline for 3D Multimodality Image Integration and Computer-assisted Planning in Epilepsy Surgery
    09:41

    A Pipeline for 3D Multimodality Image Integration and Computer-assisted Planning in Epilepsy Surgery

    Published on: May 20, 2016

    12.7K
    Application of Deep Learning-Based Medical Image Segmentation via Orbital Computed Tomography
    04:48

    Application of Deep Learning-Based Medical Image Segmentation via Orbital Computed Tomography

    Published on: November 30, 2022

    3.3K

    Area of Science:

    • Medical Imaging
    • Artificial Intelligence
    • Computer Vision

    Background:

    • Medical imaging often involves multiple modalities (e.g., MRI, ultrasound).
    • Missing data in multimodal medical imaging presents a significant challenge for diagnosis and treatment planning.
    • Existing methods struggle with high-resolution image synthesis and fusing incomplete multimodal information.

    Purpose of the Study:

    • To develop a deep learning model capable of synthesizing missing images from observed multimodal data.
    • To address challenges in generating high-resolution images, estimating missing information, fusing multimodal data, and handling incomplete datasets.
    • To improve cross-modal image synthesis for pre-operative and intra-operative medical imaging.

    Main Methods:

    • Proposed a deep mixture of multimodal hierarchical variational auto-encoders (MMHVAE).
    • MMHVAE creates complex latent representations for high-resolution image generation.
    • Employed variational distributions to estimate missing information for cross-modal synthesis.
    • Incorporated dataset-level information to manage incomplete training data.

    Main Results:

    • MMHVAE successfully synthesized missing images from observed multimodal data.
    • The model demonstrated effectiveness in generating high-resolution images.
    • Achieved robust performance in cross-modal image synthesis using incomplete datasets.
    • Validated on challenging pre-operative brain multi-parametric MRI and intra-operative ultrasound imaging.

    Conclusions:

    • The proposed MMHVAE model offers a powerful solution for synthesizing missing multimodal medical images.
    • This approach enhances the utility of incomplete medical imaging datasets for clinical applications.
    • MMHVAE shows promise for improving diagnostic accuracy and treatment planning through advanced image reconstruction.