Using the Relative Entropy of Linguistic Complexity to Assess L2 Language Proficiency Development.

Area of Science:

• Computational Linguistics
• Second Language Acquisition
• Corpus Linguistics

Background:

• Assessing second language (L2) proficiency development requires robust analytical methods.
• Traditional frequency-based analyses of L2 corpora often lack regularity in detecting proficiency patterns.
• Existing methods may not fully capture the nuanced progression of L2 learners.

Purpose of the Study:

• To apply relative entropy as a novel method for quantifying differences among L2 learners at various proficiency levels.
• To investigate the utility of relative entropy in detecting patterns of lexical and grammatical development in L2 acquisition.
• To compare the effectiveness of relative entropy against traditional statistical methods for analyzing L2 proficiency.

Main Methods:

• Utilized a large-scale naturalistic corpus of second language (L2) data.
• Employed relative entropy to analyze linguistic features including lemma, token, POS-trigram, and conjunction.
• Compared relative entropy results with those obtained from traditional time series and frequency-based methods.

Main Results:

• Information distribution discrimination, reflecting lexical and grammatical differences, significantly increases from lower to higher L2 proficiency levels.
• Relative entropy analysis revealed clear patterns of increasing linguistic complexity and diversity in L2 learners.
• Traditional frequency-based methods showed limited regularity in detecting L2 proficiency development compared to relative entropy.

Conclusions:

• Relative entropy is an effective and practical algorithm for stably detecting and predicting L2 proficiency development.
• The findings support the hypothesis that L2 learners progress towards more complex and diverse language use.
• Relative entropy offers a superior approach to traditional methods for analyzing L2 learner data and understanding acquisition trajectories.