Related Experiment Video
Updated: Aug 6, 2026
10:41
VisualEyes: A Modular Software System for Oculomotor Experimentation
Published on: March 25, 2011
A Modular Synthetic Data Generation Toolkit for Vision-Based Localization Models in Intraocular Robotic Microsurgery
Siying Zhu1, Yub Heo1, Mojtaba Esfandiari1
1Laboratory for Computational Sensing and Robotics, Johns Hopkins University, Baltimore, MD 21218, USA.
Journal of Medical Robotics Research
|July 17, 2026
Summary
A new simulation framework aids robotic eye surgery by creating realistic data for vision-based localization. This enhances control precision for cable-driven continuum robots like the Improved Integrated Robotic Intraocular Snake (I²RIS).
Area of Science:
- Robotics
- Ophthalmology
- Computer Vision
Background:
- Intraocular microsurgery demands extreme precision in a confined space.
- Cable-driven continuum robots offer dexterity but suffer from nonlinear hysteresis, hindering precise control and localization.
- Vision-based localization is crucial for compensating robotic control uncertainties.
Purpose of the Study:
- To establish an open-source simulation framework for developing and evaluating vision-based localization methods in ophthalmic surgery.
- To generate a diverse dataset of synchronized RGB-D images and 6D pose data for training and testing localization algorithms.
- To assess the sim-to-real transferability of vision-based approaches for intraocular continuum robotics.
Main Methods:
- Developed a modular, open-source simulation environment replicating an ophthalmic surgical scene with an eyeball model, I²RIS robot, and surgical microscope.
- Acquired synchronized RGB-D images and ground-truth 6D pose data under varied conditions.
- Conducted experiments on geometry-driven 6D pose tracking and appearance-based sim-to-real detection.
- Performed domain gap analysis using Canny edge statistics and SSIM to compare simulated and real images.
Main Results:
- Generated a comprehensive dataset for vision-based localization in intraocular robotics.
- Demonstrated the utility of the simulation framework for evaluating localization algorithms.
- Quantified the visual alignment between simulated and real microscope images, indicating good sim-to-real transferability.
Conclusions:
- The established simulation framework provides a valuable resource for reproducible research in intraocular continuum robotics.
- The generated dataset supports the development and validation of vision-based localization techniques, enhancing robotic control precision.
- This work facilitates sim-to-real generalization for robotic surgery applications.