Related Experiment Video
Updated: May 22, 2026
09:26
Combining Augmented Reality and 3D Printing to Display Patient Models on a Smartphone
Published on: January 2, 2020
Informatics in radiology: Intuitive user interface for 3D image manipulation using augmented reality and a smartphone
Norio Nakata1, Naoki Suzuki, Asaki Hattori
1Department of Radiology and Institute for High Dimensional Medical Imaging, Research Center for Medical Sciences, Jikei University School of Medicine, 3-25-8 Nishi-Shinbashi, Minato-ku, Tokyo 1058461, Japan. nakata@jikei.ac.jp
Summary
A new augmented reality (AR) system uses a smartphone as a remote control for manipulating complex three-dimensional (3D) and four-dimensional (4D) medical images. This technology offers intuitive 3D and 4D image interaction for radiologists.
Area of Science:
- Computer Science
- Medical Imaging
- Human-Computer Interaction
Background:
- Traditional computer mice are inadequate for complex three-dimensional (3D) image manipulation.
- Augmented reality (AR) offers a novel paradigm for interactive 3D user interfaces (UI).
- There is a need for intuitive tools for manipulating advanced medical imaging data.
Purpose of the Study:
- To develop and assess a novel system for 3D and four-dimensional (4D) image manipulation using AR and a smartphone remote.
- To evaluate the usability of the AR system for radiologists in manipulating medical images.
Main Methods:
- Developed an AR system integrating optical tracking with a smartphone as a remote control.
- The smartphone, housed in a case with a fiducial marker, connects via WiFi to a PC with a webcam.
- The smartphone's touchscreen UI facilitates 3D and 4D image manipulation in an AR environment.
Main Results:
- The system enables radiologists to easily manipulate 3D and 4D computed tomography (CT) and magnetic resonance imaging (MRI) data.
- High-quality image resolution is maintained within the AR environment.
- Pilot assessments indicate feasibility for future clinical use in reading rooms.
Conclusions:
- The smartphone-integrated AR system provides an effective solution for 3D and 4D medical image manipulation.
- This technology has the potential to enhance radiological workflow and interpretation.
- Further implementation in clinical settings is anticipated in the near future.
