The evolution of a visual-to-auditory sensory substitution device using interactive genetic algorithms

Area of Science:

• Neuroscience
• Human-Computer Interaction
• Perception Science

Background:

• Sensory substitution devices (SSDs) offer a way to compensate for sensory loss by translating information between modalities.
• Designing effective SSDs is challenging due to the vast possibilities for sensory information conversion and the need to consider human perceptual limits and user preferences.

Purpose of the Study:

• To develop design guidelines for improved sensory substitution devices (SSDs).
• To explore the use of interactive genetic algorithms for optimizing SSD design.
• To identify optimal sensory mappings based on auditory aesthetics and perceptual performance.

Main Methods:

• An interactive genetic algorithm approach was adapted to explore a wide range of potential sensory substitution device designs.
• Three experiments were conducted to evaluate device fitness based on auditory aesthetics, sound-image matching accuracy, and perceptual discrimination.
• The algorithm iteratively selected and bred the most successful device configurations over multiple generations.

Main Results:

• The genetic algorithm successfully identified and selected for specific traits in SSDs based on different fitness criteria.
• Experiment 1 showed traits selected for auditory aesthetics, while Experiments 2 and 3 revealed traits optimized for perceptual tasks like sound-image matching and discrimination.
• The selected traits provide empirical evidence for optimizing sensory mapping in SSDs.

Conclusions:

• Interactive genetic algorithms are effective tools for exploring the design space of sensory substitution devices.
• The study provides data-driven design guidelines for creating more effective and user-friendly SSDs.
• Optimizing SSDs based on both perceptual performance and user aesthetics is crucial for successful sensory substitution.