Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Looking Beyond Two Frames: End-to-End Multi-Object Tracking Using Spatial and Temporal Transformers.

IEEE transactions on pattern analysis and machine intelligence·2022
Same author

Bio-conductivity characteristics of chronic kidney disease stages examined by portable frequency-difference electrical impedance tomography.

Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference·2022
Same author

Leveraging Regular Fundus Images for Training UWF Fundus Diagnosis Models via Adversarial Learning and Pseudo-Labeling.

IEEE transactions on medical imaging·2021
Same author

Events, Event Prediction, and Predictive Processing.

Topics in cognitive science·2020
Same author

CMOS stimulating chips capable of wirelessly driving 473 electrodes for a cortical vision prosthesis.

Journal of neural engineering·2019
Same author

Approximate Fisher Information Matrix to Characterize the Training of Deep Neural Networks.

IEEE transactions on pattern analysis and machine intelligence·2018
Same journal

Analysis of End-Tidal CO2 Variability During Plateau Waves Episodes: An Information Theoretic Approach<sup></sup>.

Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference·2025
Same journal

AI and Tomosynthesis for Breast Cancer Molecular Subtyping: A step toward precision medicine<sup></sup>.

Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference·2025
Same journal

Towards Sustainable Protein Recovery from Biological Waste: Assessing Polyethersulfone-based Microfiltration.

Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference·2025
Same journal

Analysis of the cardiovascular response to standardized polymicrobial peritonitis experimental model.

Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference·2025
Same journal

Automated Wrist Ultrasound Image Bone Enhancement and Segmentation Using Deep Learning.

Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference·2025
Same journal

A Deep Learning approach for Depressive Symptoms assessment in Parkinson's disease patients using facial videos.

Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference·2025
See all related articles

Related Experiment Video

Updated: May 14, 2026

Characterization of the Sense of Agency over the Actions of Neural-machine Interface-operated Prostheses
05:21

Characterization of the Sense of Agency over the Actions of Neural-machine Interface-operated Prostheses

Published on: January 7, 2019

Transformative Reality: improving bionic vision with robotic sensing.

Wen Lik Dennis Lui1, Damien Browne, Lindsay Kleeman

  • 1Monash Vision Group, Monash Unjversity, Clayton, Australia. Wen.Lui@monash.edu

Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference
|February 1, 2013
PubMed
Summary
This summary is machine-generated.

This article introduces Transformative Reality, a new approach to improving vision for people using bionic eye implants. By combining camera data with other sensors and advanced robotic modeling, the system helps users better navigate and interact with their surroundings despite the low resolution of current prosthetic devices.

Keywords:
prosthetic eyessensory substitutionimage processinghuman-computer interaction

Frequently Asked Questions

More Related Videos

Techniques for Processing Eyes Implanted with a Retinal Prosthesis for Localized Histopathological Analysis: Part 2 Epiretinal Implants with Retinal Tacks
10:00

Techniques for Processing Eyes Implanted with a Retinal Prosthesis for Localized Histopathological Analysis: Part 2 Epiretinal Implants with Retinal Tacks

Published on: February 14, 2015

Haptic/Graphic Rehabilitation: Integrating a Robot into a Virtual Environment Library and Applying it to Stroke Therapy
13:44

Haptic/Graphic Rehabilitation: Integrating a Robot into a Virtual Environment Library and Applying it to Stroke Therapy

Published on: August 8, 2011

Related Experiment Videos

Last Updated: May 14, 2026

Characterization of the Sense of Agency over the Actions of Neural-machine Interface-operated Prostheses
05:21

Characterization of the Sense of Agency over the Actions of Neural-machine Interface-operated Prostheses

Published on: January 7, 2019

Techniques for Processing Eyes Implanted with a Retinal Prosthesis for Localized Histopathological Analysis: Part 2 Epiretinal Implants with Retinal Tacks
10:00

Techniques for Processing Eyes Implanted with a Retinal Prosthesis for Localized Histopathological Analysis: Part 2 Epiretinal Implants with Retinal Tacks

Published on: February 14, 2015

Haptic/Graphic Rehabilitation: Integrating a Robot into a Virtual Environment Library and Applying it to Stroke Therapy
13:44

Haptic/Graphic Rehabilitation: Integrating a Robot into a Virtual Environment Library and Applying it to Stroke Therapy

Published on: August 8, 2011

Area of Science:

  • Biomedical engineering and bionic vision research
  • Robotic sensing and computer vision systems

Background:

Current visual prostheses offer limited spatial and intensity resolution compared to natural human sight. This technological gap restricts the utility of existing implants for daily tasks. Prior research has shown that standard image processing struggles to convey complex environmental information through low-resolution phosphene arrays. That uncertainty drove the development of more sophisticated sensory integration techniques. No prior work had resolved how to effectively represent intricate objects within such constrained visual feedback. This gap motivated the exploration of multi-modal data streams for prosthetic enhancement. Researchers have long sought methods to preserve salient features like edges in these restricted displays. The field remains challenged by the inherent difficulty of translating high-fidelity camera input into meaningful prosthetic signals.

Purpose Of The Study:

The aim of this study is to improve the functionality of bionic vision through a new approach called Transformative Reality. Current prosthetic devices suffer from extremely low spatial and intensity resolution, which hinders user performance. This research addresses the challenge of converting complex camera video into meaningful visual signals for implants. The authors seek to preserve salient environmental features that are often lost during standard image processing. By incorporating non-visual sensors, the team intends to detect objects that cameras alone cannot identify. The study also explores how real-time world modeling can assist in representing complex entities like people. Researchers want to determine if these advanced techniques can enable tasks such as indoor navigation and object manipulation. This work is motivated by the need to make prosthetic vision more practical for daily life.

Main Methods:

The review approach involved evaluating a novel framework for enhancing prosthetic visual feedback. Investigators utilized a head-mounted display to mimic the constraints of a 25 by 25 binary phosphene array. This experimental setup allowed for controlled testing of visual processing improvements. The team integrated diverse sensor inputs to provide comprehensive environmental data. Robotic algorithms were employed to generate real-time spatial models of the surroundings. Researchers compared this new methodology against standard camera-based image processing techniques. Simulated trials focused on assessing performance during specific navigation and manipulation tasks. Data collection centered on the ability of subjects to identify objects and navigate indoor spaces effectively.

Main Results:

The strongest finding indicates that the proposed framework supports functional tasks in environments where traditional processing is ineffective. Preliminary trials show that subjects can successfully perform indoor navigation and object manipulation. The system enables the detection of complex features such as people and navigable ground. By using a 25 by 25 phosphene display, the researchers demonstrated that their rendering techniques improve visual representation. The results suggest that multi-modal data integration overcomes limitations inherent in camera-only systems. The study reports that these enhancements are achievable in real time. The authors found that the combination of visual and non-visual sensors provides superior environmental awareness. These findings highlight the potential for significant improvements in prosthetic utility.

Conclusions:

The authors propose that Transformative Reality enhances functional performance for prosthetic users. Synthesis and implications suggest that integrating non-visual sensors improves environmental awareness beyond standard camera-based systems. The researchers indicate that real-time world modeling facilitates the identification of navigable paths and human presence. This approach demonstrates that visually rendering complex entities allows for better representation in low-resolution formats. The study suggests that these methods enable tasks like indoor navigation and object manipulation where traditional processing fails. The authors conclude that their framework provides a viable pathway for improving bionic vision utility. These findings imply that multi-modal sensing is a key factor for future prosthetic design. The team maintains that their simulated trials support the potential for real-world prosthetic application.

The researchers propose that the system improves navigation and object manipulation by combining multi-modal sensor data with real-time world modeling. This allows the prosthetic to represent complex entities like people more effectively than standard camera-only processing, which often fails in cluttered environments.

The framework utilizes a combination of visual cameras and non-visual sensors to gather environmental data. These inputs are processed by robotic algorithms to construct real-time models of the surroundings, which are then rendered into a format suitable for the 25x25 phosphene display.

The researchers state that the head-mounted display is necessary to simulate the 25x25 binary phosphene resolution of current implants. This constraint ensures that the testing environment accurately reflects the limitations of existing bionic vision hardware during the preliminary trials.

The system uses robotic sensing algorithms to process multi-modal data. This role is critical for constructing real-time world models, which allow the software to detect complex features like empty ground or human subjects that are otherwise difficult to distinguish in low-resolution imagery.

The researchers measured performance through simulated trials involving indoor navigation, object manipulation, and people detection. Subjects using the display were tested on their ability to complete these tasks, with the authors reporting success in scenarios where traditional processing methods were found to be unusable.

The authors propose that their multi-modal approach provides a functional advantage over traditional methods. They claim that by extending sensory input and improving rendering, the system overcomes the resolution limitations that currently restrict the utility of implanted visual prostheses for patients.