Localizing category-related information in speech with multi-scale analyses.

Area of Science:

• Speech processing
• Computational linguistics
• Machine learning

Background:

• Speech production involves high-dimensional physical signals (vocal tract geometry, acoustic energy).
• Linguistic theories propose low-dimensional categories (phonemes, phrase types).
• Quantifying information about categories in speech signals is challenging.

Purpose of the Study:

• Develop a method to localize category-related information in speech signals.
• Investigate the temporal distribution of phonemic/gestural and syntactic information.
• Compare the effectiveness of different machine learning algorithms for this task.

Main Methods:

• A multi-scale analysis approach using machine learning algorithms.
• Systematically restricting the temporal extent of training input to assess classification accuracy.
• Examining linear discriminant analysis (LDA) and long short-term memory (LSTM) neural networks.
• Analyzing phonemic/gestural and syntactic relative clause categories.

Main Results:

• Both LDA and LSTM detected category-related information earlier and later than standard linguistic theories predict.
• LSTM neural networks identified category-related information more effectively than LDA.
• The method successfully localized information related to both phonemic and syntactic categories.

Conclusions:

• Linguistic information is distributed differently in speech signals than traditionally assumed.
• Machine learning, particularly LSTMs, offers powerful tools for analyzing speech information.
• This approach advances our understanding of the relationship between linguistic theory and acoustic speech signals.