HEARTBEAT4D: An Open-source Toolbox for Turning 4D Cardiac CT into VR/AR

Area of Science:

• Medical Imaging
• Computer Science
• Biomedical Engineering

Background:

• Four-dimensional (4D) imaging data is increasingly prevalent in medical diagnostics like CT and MRI.
• Current clinical visualization often examines static temporal phases, potentially missing insights from dynamic physiological motion.
• Existing methods lack open-source tools for converting 4D medical imaging into interactive augmented reality (AR) or virtual reality (VR) experiences.

Purpose of the Study:

• To develop an accessible, open-source workflow for transforming 4D cardiac CT data into dynamic AR visualizations.
• To enable interactive exploration of cardiac motion and anatomical structures using readily available devices.
• To address the gap in tools for advanced visualization of 4D medical imaging data.

Main Methods:

• A workflow was established using free and open-source software (3D Slicer and Unity).
• Custom software was developed to automatically propagate segmentations across cardiac phases and export surface files.
• A user interface and code were created for animating and interactively reviewing 3D surfaces in AR.

Main Results:

• The workflow successfully processes 4D cardiac CT data from DICOM to dynamic AR representations.
• Automated segmentation propagation and AR review features were successfully implemented.
• Surface-based area measurements showed excellent correlation (R > 0.99) with radiologist assessments.

Conclusions:

• The developed open-source workflow provides a novel method for visualizing 4D cardiac CT data in AR.
• The framework is adaptable for 4D imaging of other tissues and modalities, offering a blueprint for future applications.
• These tools hold significant potential for clinical diagnosis, surgical planning, and medical education.