Health-Related Content in Transformer-Based Deep Neural Network Language Models: Exploring Cross-Linguistic Syntactic...

Area of Science:

• Natural Language Processing
• Artificial Intelligence
• Computational Linguistics

Background:

• Transformer-based deep neural networks are increasingly used for language tasks.
• Assessing and mitigating bias in these models is crucial for reliable information dissemination.
• Previous bias studies often focused on semantic or knowledge-based biases.

Purpose of the Study:

• To develop and apply a methodology for quantifying bias in transformer language models.
• To investigate the nature of bias (semantic vs. syntactic) in multilingual models (Chinese, English, French).
• To examine bias specifically within the context of health-related mythbusters concerning COVID-19.

Main Methods:

• Utilized transformer-based deep neural network language models.
• Employed a methodology for bias quantification.
• Tested models with queries related to COVID-19 health mythbusters across multiple languages.

Main Results:

• Observed a bias in the models' responses to health-related queries.
• Identified the bias as primarily syntactic, not semantic or knowledge-based.
• Results align with theoretical predictions regarding structural complexity and bias.

Conclusions:

• The study underscores the presence of syntactic bias in large language models.
• There is a critical need for developing specialized health-communication corpora for training deep learning models.
• Future research should focus on creating and utilizing such corpora to improve model fairness and accuracy in health contexts.