Audio drawing and working memory in blindness: design, development and validation of a multi-sensory system

Area of Science:

• Cognitive Neuroscience
• Human-Computer Interaction
• Sensory Rehabilitation

Background:

• Working memory (WM) is vital for daily functioning, yet its multi-sensory aspects, particularly in congenital blindness, remain under-explored due to technological limitations.
• Existing research on WM predominantly focuses on visual memory, leaving a gap in understanding cross-modal integration and spatial representation in individuals with visual impairments.

Purpose of the Study:

• To develop and evaluate a novel audio-haptic system for investigating multi-sensory spatial working memory.
• To examine the generation and recall of spatial representations in both visually impaired and sighted individuals using the developed technology.
• To assess the potential of this technology for studying sensory influences on cognition and the effects of early visual deprivation.

Main Methods:

• Development of an integrated audio-haptic system comprising an audio-tactile tablet with spatialized speakers, tactile sensors, and tri-modal (acoustic, visual, haptic) feedback units.
• Utilizing the system to present and assess spatial memory tasks involving multi-sensory stimuli.
• Recruiting visually impaired and sighted adult participants to interact with and evaluate the system's usability.

Main Results:

• The developed audio-haptic system was successfully integrated and demonstrated ease of interaction for both visually impaired and sighted adult users.
• The technology provides a viable platform for delivering multi-sensory spatial stimuli and assessing working memory recall.
• Preliminary findings suggest the system is suitable for experimental and clinical applications.

Conclusions:

• The novel audio-haptic system effectively supports the study of multi-sensory working memory and spatial representation across different sensory abilities.
• This technology holds promise for advancing research into high-level cognitive skills and informing rehabilitative interventions for individuals with visual impairments.
• Further research using this system can elucidate the impact of sensory modalities and early visual deprivation on cognitive functions.