The cross-linguistic performance of word segmentation models over time

Area of Science:

• Computational Linguistics
• Psycholinguistics
• Language Acquisition

Background:

• Automated word segmentation is crucial for analyzing child language development.
• Existing models vary in their ability to handle diverse linguistic inputs.
• Psycholinguistic principles offer a foundation for improving segmentation accuracy.

Purpose of the Study:

• To compare the performance of three psycholinguistically-informed word segmentation models.
• To identify linguistic factors contributing to cross-linguistic variation in segmentation accuracy.
• To establish a foundation for future research on challenges in child language segmentation.

Main Methods:

• Evaluated three models (transitional probabilities, diphone-based, PUDDLE) on 132 CHILDES corpora (28 languages, 11.9M words).
• Assessed model performance based on caregiver utterances.
• Utilized regression analysis to correlate segmentation performance with linguistic features.

Main Results:

• PUDDLE demonstrated superior overall performance compared to the other models.
• Significant cross-linguistic variation in segmentation performance was observed.
• Linguistic properties including word length, lexical diversity, and phoneme patterns explained 40% of performance variation.

Conclusions:

• PUDDLE is a robust model for word segmentation across diverse languages.
• Lexico-phonological characteristics significantly influence the difficulty of word segmentation.
• Further research is needed to explore additional variables affecting segmentation performance.