Race, Ethnicity and Their Implication on Bias in Large Language Models

Area of Science:

• Artificial Intelligence
• Natural Language Processing
• Computational Linguistics

Background:

• Large language models (LLMs) are increasingly used in sensitive areas like healthcare.
• Existing research highlights outcome disparities but lacks insight into internal LLM mechanisms.
• Understanding how LLMs represent demographic attributes is crucial for mitigating bias.

Purpose of the Study:

• To investigate the internal mechanisms by which large language models represent and operationalize race and ethnicity.
• To analyze the distribution and function of demographic information within LLM architectures.
• To assess the impact of targeted interventions on bias reduction.

Main Methods:

• Analysis of three open-source LLMs using two public datasets (toxicity generation, clinical narrative understanding).
• Application of a reproducible interpretability pipeline combining probing, neuron-level attribution, and targeted intervention.
• Examination of how demographic cues influence model behavior and internal representations.

Main Results:

• Demographic information is distributed across internal model units with significant cross-model variability.
• Some units exhibit stereotype-associated learning from pretraining data.
• Identical demographic cues can trigger different model behaviors, and interventions yield incomplete bias reduction.

Conclusions:

• LLM internal representations of race and ethnicity are complex and vary across models.
• Bias mitigation strategies targeting specific neurons show limited effectiveness, suggesting deeper representational challenges.
• Further research is needed for more systematic approaches to address bias in LLMs.