A Multisensory Approach to Present Phonemes as Language Through a Wearable Haptic Device

Area of Science:

• Human-computer interaction
• Sensory substitution
• Wearable technology

Background:

• Communication can be impaired by sensory deficits or overloaded channels.
• The sense of touch offers an alternative sensory modality for information transfer.
• Multi-sensory haptic cues are promising for communication applications requiring extensive, discrete cue sets in a compact form.

Purpose of the Study:

• To present a novel multi-sensory haptic device for language transmission.
• To encode phonemes as distinct haptic cues using vibration, squeeze, and stretch.
• To evaluate the efficacy of haptic communication through user training and word recognition.

Main Methods:

• Development of a small, wearable multi-sensory haptic device for the upper arm.
• Encoding phonemes into haptic cues comprising vibration, radial squeeze, and lateral skin stretch.
• Training participants over four to eight days to recognize haptically transmitted phonemes and words.

Main Results:

• Participants achieved 89% accuracy in word identification using multiple choice after four days of training.
• A subset of participants reached 70% accuracy in free-response word recognition after eight days.
• The study demonstrates high word recognition performance with a compact, wearable haptic device.

Conclusions:

• Multi-sensory haptics show significant promise for effective haptic communication.
• Wearable haptic technology can facilitate language transmission, even with sensory impairments.
• This approach offers a viable alternative communication channel when standard methods are unavailable.