A Behavioral Study of Event-based Depth-Filtered Prosthetic Vision in Simulated Dynamic Environments

Area of Science:

• Biomedical Engineering
• Neuroscience
• Computer Vision

Background:

• Cortical visual prostheses create artificial vision via electrical stimulation, but current electrode arrays are limited by size, restricting visual information.
• Phosphenes are light perceptions elicited by these prostheses, crucial for restoring sight.

Purpose of the Study:

• To investigate a lightweight disparity-based depth filter for improving task performance in simulated phosphene vision.
• To evaluate the algorithm's effectiveness in a simulated road-crossing task using sighted subjects.
• To assess if a reinforcement learning (RL) agent can serve as a human surrogate for future algorithm development.

Main Methods:

• A behavioral study was conducted with sighted subjects using simulated phosphene vision.
• A lightweight disparity-based depth filter was applied to extract relevant pixel information from a stereo event camera.
• Subjects performed a simulated road-crossing task, and their success rates were compared with and without the depth filter.
• The performance of human subjects was benchmarked against a reinforcement learning (RL) agent.

Main Results:

• The depth filter significantly improved the task success rate of subjects from 50% to 95%.
• Increasing the phosphene count had a minor impact on performance.
• Subject success rates were only 4.8% lower than the peak accuracy achieved by the RL agent.

Conclusions:

• Disparity-based depth filtering is a promising technique for enhancing visual prosthesis performance.
• The close alignment between human and RL agent performance suggests RL can be valuable for future visual prosthesis algorithm development.
• This low-compute algorithm is suitable for portable visual prosthesis systems.