Towards biologically plausible phosphene simulation for the differentiable optimization of visual cortical prostheses

Area of Science:

• Neuroscience
• Biomedical Engineering
• Computer Science

Background:

• Millions worldwide suffer from blindness, with cortical visual prostheses offering a potential vision restoration method.
• These prostheses convert camera input into electrical stimulation, creating artificial visual percepts called phosphenes.
• Optimizing phosphene encoding is crucial for efficacy, efficiency, and practical use, often using simulated prosthetic vision (SPV) pipelines.

Purpose of the Study:

• To present a biologically plausible, real-time phosphene simulator for cortical visual prostheses.
• To enable gradient-based computational optimization of phosphene encoding models using differentiable operations.
• To provide a flexible simulation framework for visual neuroprosthetics research.

Main Methods:

• Developed a PyTorch-based phosphene simulator integrating clinical and neurophysiological data.
• Incorporated models of visual cortex retinotopic organization and cortical magnification.
• Included quantitative effects of stimulation parameters and temporal dynamics on phosphene characteristics.

Main Results:

• The simulator achieves real-time performance and biological plausibility.
• It supports gradient-based optimization of encoding models via differentiable operations.
• Demonstrated suitability for deep learning-based prosthetic vision optimization and behavioral experiments.

Conclusions:

• The open-source phosphene simulator offers a flexible framework for visual neuroprosthetics research.
• It facilitates computational, clinical, and behavioral neuroscience applications.
• Aids in advancing the development and optimization of cortical visual prostheses.