Considering Performance in the Automated and Manual Coding of Sociolinguistic Variables: Lessons From Variable (ING)

Area of Science:

• Sociolinguistics
• Computational Linguistics
• Speech Processing

Background:

• Impressionistic coding of sociolinguistic variables, such as English (ING) pronunciation, is crucial for language variation and change studies.
• Automating sociolinguistic data analysis has advanced, but coding features like (ING) without clear acoustic correlates lags behind other phonetic features.

Purpose of the Study:

• To explore computational methods for automatically coding variable (ING) pronunciation in speech recordings.
• To compare automated coding performance with human coders and assess the impact on linguistic and social factor analyses.

Main Methods:

• Utilized automatic speech recognition (ASR) procedures, specifically forced alignment with the Montreal Forced Aligner.
• Employed supervised machine learning algorithms, including linear and radial support vector machines, and random forests.
• Assessed algorithm performance through agreement with human coders and impact on sociolinguistic analysis outcomes.

Main Results:

• Automated coding methods demonstrate potential for analyzing sociolinguistic variables like (ING).
• Variability in results is expected, even with meticulously human-coded data, underscoring the importance of robust validation.
• Baseline performance data for both automated and manual coding methods were established.

Conclusions:

• Automated coding of sociolinguistic variables like (ING) is feasible and promising.
• Further research is needed to establish "gold standards" for training and testing automated procedures.
• The study provides valuable baseline data and highlights considerations for integrating automated methods into sociolinguistic research.