Simulation of prosthetic vision with the PRIMA system and enhancement of face representation

Patients implanted with the PRIMA photovoltaic subretinal prosthesis in geographic atrophy report form vision with the average acuity matching the 100 μm pixel size. Although this remarkable outcome enables them to read and write, they report difficulty with perceiving faces. Despite the pixelated stimulation, patients report seeing smooth patterns rather than dots. This paper provides a novel, non-pixelated algorithm for simulating prosthetic vision the way it is experienced by PRIMA patients, compares the algorithm's predictions to clinical perceptual outcomes, and offers computer vision and machine learning (ML) methods to improve face representation.

Our simulation algorithm (ProViSim) integrates a grayscale filter, spatial resolution filter, and contrast filter. This accounts for the limited sampling density of the retinal implant (pixel pitch), as well as the reduced contrast sensitivity of prosthetic vision. Patterns of Landolt C and faces created using this simulator are compared to reports from actual PRIMA users. To recover the facial features lost in prosthetic vision due to limited resolution or contrast, we apply an ML facial landmarking model, as well as contrast-adjusting tone curves to the image prior to its projection onto the photovoltaic retinal implant.

Prosthetic vision simulated using the above algorithm matches the maximum letter acuity observed in clinical studies, as well as the patients' subjective descriptions of perceived facial features. Applying the inversed contrast filter to the image prior to its projection onto the implant and accentuating the facial features using an ML facial landmarking model helps preserve the contrast in prosthetic vision, improves emotion recognition and reduces the response time.

Spatial and contrast constraints of prosthetic vision limit resolvable features and degrade natural images. ML based methods and contrast adjustments prior to image projection onto the implant mitigate some limitations and improve face representation. Even though higher spatial resolution can be expected with implants having smaller pixels, contrast enhancement still remains essential for face recognition.