A Behavioral Study of Event-based Depth-Filtered Prosthetic Vision in Simulated Dynamic Environments

Cortical visual prostheses elicit light perceptions called phosphenes by electrically stimulating an implanted electrode array. One severe limitation with current arrays is the size of the electrode array and therefore, the reduced visual information that can be presented. This paper investigates the application of a lightweight disparity-based depth filter to improve task performance. The algorithm extracts information from pixels of a stereo event camera within a specific disparity range, thereby removing unnecessary information. The low-compute algorithm is also attractive for future portable systems. We report on a behavioral study of sighted subjects using simulated phosphene vision to perform a road-crossing task within a simulated environment and evaluate the impact of the depth filtering on their success rate. We compare the subjects' performance with the performance of a reinforcement learning (RL) agent to determine if it can act as a human surrogate for future algorithm development. Our results show that the use of the depth filter significantly improved the task success rate of the subjects from 50% to 95%. The effect of an increased phosphene count on the performance was minor. The success rate of the subjects was only 4.8% below the peak accuracy of the RL agent in the same experimental scenarios. The close alignment of the RL agent and human performance results, supports the possibility of using primarily RL studies in further prosthesis algorithmic development.