The sociolinguistic foundations of language modeling.

Area of Science:

• Computational Linguistics
• Sociolinguistics

Background:

• Language models (LMs) are increasingly powerful but face challenges.
• A sociolinguistic perspective offers a novel framework for understanding LMs.

Purpose of the Study:

• To introduce a sociolinguistic perspective on language modeling.
• To explore how understanding language varieties can address LM challenges.

Main Methods:

• Defining "language variety" from sociolinguistics.
• Analyzing LM challenges (social bias, domain adaptation, alignment, language change, scale) through this lens.

Main Results:

• Language models inherently model "language varieties."
• This perspective illuminates key LM challenges.

Conclusions:

• Sociolinguistics provides valuable insights for LM development and deployment.
• Carefully curated training corpora representing specific language varieties are crucial for LM performance and societal value.