Glove-Talk: a neural network interface between a data-glove and a speech synthesizer

Area of Science:

• * Artificial Intelligence
• * Human-Computer Interaction
• * Neural Networks

Background:

• * Adaptive interfaces are crucial for personalized user experiences.
• * Developing complex mappings for interfaces like gesture-to-speech requires advanced computational models.

Purpose of the Study:

• * To explore the potential of multilayer neural networks in creating adaptive interfaces.
• * To implement and evaluate a hand-gesture to speech system using neural networks.

Main Methods:

• * A VPL Data-Glove and DECtalk speech synthesizer were connected via five neural networks.
• * A modified backpropagation learning procedure was used for training on a single speaker's data.
• * The system was trained with a 203 gesture-to-word vocabulary.

Main Results:

• * The system achieved high accuracy, with errors less than 1% and no word production about 5% of the time.
• * Adaptive control of speaking rate and word stress was successfully implemented.
• * Using smaller, specialized networks improved training times and performance speed.

Conclusions:

• * Multilayer neural networks are effective for learning complex mappings in adaptive interfaces.
• * The developed system demonstrates the feasibility of high-bandwidth, user-adaptive interfaces.
• * Neural networks offer a powerful approach for creating personalized communication tools.