Self-Attention Based Vision Processing for Prosthetic Vision.

Area of Science:

• Computer Vision
• Biomedical Engineering
• Artificial Intelligence

Background:

• Prosthetic vision systems aim to restore visual capabilities.
• Current methods often require manual feature selection and post-processing.
• Learning-based approaches for prosthetic vision lack robust interpretability.

Purpose of the Study:

• To investigate Self-Attention (SA) networks for direct visual representation learning in prosthetic vision.
• To leverage SA for task-specific scene representations, eliminating the need for hand-selected features.
• To utilize SA's importance mapping as an explainability tool for prosthetic vision systems.

Main Methods:

• Employing Self-Attention networks to learn visual representations directly from input.
• Developing task-specific scene representations for prosthetic vision applications.
• Mapping attention weights to image regions for analyzing model behavior.
• Evaluating the approach within an orientation and mobility (OM) task context.

Main Results:

• Demonstrated the feasibility of SA networks for learning prosthetic vision pipelines.
• Showcased SA's ability to generate task-specific representations without manual feature engineering.
• Validated the use of attention mapping as an effective explainability tool.
• Achieved enhanced interpretation of learned vision processing behaviors compared to existing methods.

Conclusions:

• Self-Attention networks offer a promising direction for prosthetic vision research.
• The proposed method enhances interpretability and validation of prosthetic vision systems.
• This approach facilitates the development of more effective and understandable artificial vision.