Spatio-temporal articulatory movement primitives during speech production: extraction, interpretation, and validation

Area of Science:

• Speech Production and Acoustics
• Computational Linguistics
• Machine Learning for Signal Processing

Background:

• Understanding the motor control of speech production is complex.
• Decomposing speech movements into fundamental units (primitives) can elucidate underlying mechanisms.
• Existing methods may not fully capture the spatiotemporal dynamics of articulation.

Purpose of the Study:

• To develop and validate a computational approach for deriving interpretable movement primitives from speech articulation data.
• To apply a novel algorithm, convolutive Nonnegative Matrix Factorization with sparseness constraints (cNMFsc), to articulatory data.
• To quantitatively and qualitatively assess the algorithm's ability to recover linguistically relevant structures.

Main Methods:

• Utilized convolutive Nonnegative Matrix Factorization with sparseness constraints (cNMFsc).
• Decomposed speech articulation data (electromagnetic articulography and synthetic) into spatiotemporal basis sequences and activation matrices.
• Optimized a cost function balancing data reconstruction accuracy and temporal sparsity of active primitives.
• Evaluated performance using pseudo ground-truth primitives from an articulatory synthesizer based on Articulatory Phonology.

Main Results:

• The cNMFsc algorithm successfully extracted movement primitives from human speech production data.
• The derived primitives demonstrated linguistic interpretability.
• Quantitative and qualitative assessments confirmed the algorithm's ability to recover compositional structure.

Conclusions:

• The proposed computational framework effectively extracts interpretable movement primitives from speech articulation.
• This approach holds potential for advancing the understanding of speech motor control and coarticulation.
• The method offers a novel tool for analyzing complex speech dynamics.