Classification of Blind Users' Image Exploratory Behaviors Using Spiking Neural Networks

Area of Science:

• Human-Computer Interaction
• Artificial Intelligence
• Computational Neuroscience

Background:

• Blind individuals use tactile exploration for image understanding.
• Automated classification of these exploration behaviors is crucial for developing assistive technologies.
• Current methods for analyzing spatio-temporal data have limitations in accuracy and efficiency.

Purpose of the Study:

• To develop and evaluate a novel computational framework for classifying tactile image exploration procedures used by blind individuals.
• To enhance the efficiency and effectiveness of image exploration for visually impaired users through intelligent systems.

Main Methods:

• Extraction of translation-, rotation-, and scale-invariant features from user movement trajectories.
• Utilization of spiking neural networks (SNNs) for encoding numerical features into model strings.
• A distance-based classification scheme combined with Dempster-Shafter Theory (DST) for integrating feature distances and determining the final classification.

Main Results:

• The proposed framework achieved a classification accuracy of 95.89%.
• Demonstrated superior performance compared to Dynamic Time Warping (61.30%) and Hidden Markov Models (28.70%).
• Effectively encodes and classifies complex spatio-temporal data.

Conclusions:

• The developed framework provides a robust and accurate method for classifying tactile image exploration behaviors.
• It serves as a foundational component for creating intelligent interfaces to improve image exploration for the blind.
• The approach shows significant potential for advancing assistive technology in visual impairment.