Uncovering inequalities in new knowledge learning by large language models across different languages

Area of Science:

• Natural Language Processing
• Artificial Intelligence Ethics
• Computational Linguistics

Background:

• Large language models (LLMs) are increasingly vital for global productivity and problem-solving.
• Existing research on LLM linguistic inequalities focuses on static capabilities.
• LLMs' dynamic learning and knowledge acquisition necessitate investigating evolving linguistic disparities.

Purpose of the Study:

• To explore linguistic inequalities in the dynamic learning process of LLMs.
• To analyze these inequalities across four dimensions: effectiveness, transferability, prioritization, and robustness.
• To identify causes and propose mitigation strategies for emerging linguistic disparities in LLMs.

Main Methods:

• Conducted extensive experiments using in-context learning and fine-tuning settings.
• Evaluated both proprietary and open-source large language models.
• Analyzed inequalities in new knowledge acquisition across different languages and key dimensions.

Main Results:

• LLMs exhibit greater difficulty learning new knowledge efficiently and accurately in lower-resource languages.
• Knowledge transfer is more effective from higher-resource to lower-resource languages.
• New knowledge in higher-resource languages is more likely to be retained and prioritized.
• LLMs demonstrate increased robustness against misinformation in higher-resource languages.

Conclusions:

• Significant linguistic inequalities exist in LLMs' dynamic knowledge acquisition process.
• These disparities stem from linguistic factors, pretraining data, and tokenizer design.
• Addressing these inequalities, potentially through linguistic neurons, is crucial for equitable LLM development.