Learning to recognize while learning to speak: Self-supervision and developing a speaking motor.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Speech Technology

Background:

• Traditional speech synthesis and recognition are studied in isolation, hindering concurrent, lifelong learning.
• Existing systems require extensive human supervision, unlike natural infant language acquisition.

Purpose of the Study:

• To develop an autonomous learning system for concurrent speech synthesis and recognition.
• To enable agents to generate temporally-dense motor actions for speech without symbolic state pre-definition.

Main Methods:

• Treating synthesis and recognition as intertwined aspects of a lifelong learning agent.
• Utilizing self-generated, Muscles-like, High-dimensional, Temporally-dense, and Globally-smooth (MHTG) states/actions.
• Applying Candid Covariance-free Incremental (CCI) Principal Component Analysis (PCA) to develop an artificial speaking motor.

Main Results:

• Experimental results demonstrate successful learning-to-synthesize and learning-to-recognize-through-synthesis for phonemes.
• The developed Developmental Network-2 (DN-2) reaches maximum likelihood (ML) regardless of initialization, functioning as an emergent Turing Machine.
• Machine-synthesized sounds were validated through neural network and human recognition experiments.

Conclusions:

• This paradigm shift enables concurrent synthesis and recognition within a unified lifelong learning framework.
• The approach reduces the need for human supervision of motor ends, promoting autonomous learning.
• This work represents a significant step towards fully autonomous machine learning from the physical world.